Thunderdome is a tool to compare the performance of different versions of IPFS gateways by using real HTTP traffic streamed from the public gateways operated by Protocol Labs.
Thunderdome differs from other tooling such as testground because it aims to simulate realistic load conditions using close to real time requests.
A user defines an experiment that details what software they want to put under test, with any special configuration and test conditions. Each combination of software and configuration is called a target. Thunderdome deploys each target, connects them to the IPFS network and begins sending real-world requests at the chosen rate. Each target in the experiment receives exactly the same request at the same time.
Thunderdome collects metrics from each gateway and sends them to Grafana where they can be graphed and analysed. Once the experiment is done the deployed gateways are shut down to save resources, but they can be started once again if the experiment, or a variant, needs to be repeated. We use it to compare different implementations of the gateway spec, the impact of changing configuration settings or look for performance changes between releases.
Ian Davis (@Anonymous) is the DRI for Thunderdome.