Loadtesting using Gatling-FatJars and InfluxDB

For the really impatient ones: Here’s the code .

I am not getting tired talking about the importance of load testing, especially during development time. Back in summer I gave a talk about Continuous Load Testing at JUG Frankfurt, showing how one could use a combination of Gatling/Docker/Jenkins to do automated continuous load tests against specific machines.

Sadly, this approach doesn’t scale that well as it is only using a single machine for testing. Using build slaves might be another option but it starts stretching the purpose of a CI-server.

There were two problems I needed solved:

• Find a convenient way to distribute tests and their dependencies
• Collect the generated data in a central place

The solution to problem #1 was pretty obvious: FatJars. You might remember a blog post I published on that topic a while ago.

The basic idea is to package everything, the Gatling-runtime, the tests and test data, in one jar and thus make it easily distributable across a cluster of machines (e.g. Nomad by Hashicorp can distribute them directly).

The FatJar

It took a little research but I found the required parts, well hidden inside Gatling. To allow loading of data from the classpath we have to tell Gatling to use jar as the data directory.

object Engine extends App{
 val props = new GatlingPropertiesBuilder
 props.dataDirectory("jar")
 props.simulationClass(classOf[BasicSimulation].getName)
 Gatling.fromMap(props.build)
 sys.exit()
}

That’s it! Almost.

Gatling has a hard coded check for the existence of target/test-classes. As we add our tests to src/main/scala we also have to add some (empty) class in src/main/test to get around this test.

---

After a short Twitter-exchange with Stephane Landelle  he took a look at what I was doing. The good news is that the Gatling-team wants to the FatJar-option. It’s just that for the time being we have to exploit an unintended side-effect of the jar-option. Future versions of Gatling will solve this issue and the solution will probably come from this ticket.

Until then, he ensured me, the current workaround will stay in place.

---

You can now run your tests directly from the IDE. But that’s only half of what I wanted.
I added the maven-shade-plugin to my build to create the actual FatJar.

We have to use the Engine-class we just created as the Main-Class.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-shade-plugin</artifactId>
  <version>2.3</version>
  <executions>
    <execution>
      <phase>package</phase>
      <goals>
        <goal>shade</goal>
      </goals>
      <configuration>
        <filters>
          <filter>
            <artifact>*:*</artifact>
            <excludes>
              <exclude>META-INF/*.SF</exclude>
              <exclude>META-INF/*.DSA</exclude>
              <exclude>META-INF/*.RSA</exclude>
            </excludes>
          </filter>
        </filters>
        <transformers>
          <transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
            <mainClass>de.codepitbull.gatling.fatjar.Engine</mainClass>
          </transformer>
        </transformers>
      </configuration>
    </execution>
  </executions>
</plugin>

After running mvn clean package we can now do java -jar target/.jar to launch the whole test.

Adding InfluxDB

Time to collect the metrics. I decided to collect everything in InfluxDB as it supports the Graphite-protocol which in turn is directly supported by Gatling.
A simple brew install influxdb got the current version installed on my MAC. Afterwards I edited /usr/local/etc/influxdb.conf and adjusted the Graphite section:

[[graphite]]
 enabled = true
 bind-address = ":2003"
 database = "gatling"
 protocol = "tcp"

Now we can fire up InfluxDB using influxd -config /usr/local/etc/influxdb.conf.
After running the FatJar we can use influx to access the shell.

The following sequence of commands returns some of our glorious test results:

• use gatling
• show series
• select * from “gatling.basicsimulation.request_1_Redirect_1.ok.percentiles99-0”;

That’s it, time to roast your servers!