Using JMeter to measure binary protocols

In a recent project I developed a bridge component to connect a backend web service with a credit-card terminal. The terminal can only speak a binary protocol. The bridge needs to map the binary messages to the corresponding backend calls. If you are interested in details about the protocol, see this (German) Wikipedia entry for the GICC protocol.

To minimize delays at the terminal end, the customer demanded a certain level of performance. Transactions are required to complete in under one second end-to-end, i. e. including the time for the backend service to go through. An important part of the development and testing process therefore had to be performance testing and profiling to make sure that even under peak loads transactions are going to finish within the allowed time frame.

I wanted to use JMeter , but at first glance it seemed to only support “more usual” protocols without requiring me to write a custom extension. Being under a rather tight schedule, I tended towards writing a very specific measurement tool instead of learning how to extend JMeter, because I was unsure about how much effort that would be. Before doing so, however, I decided to do a little more digging into JMeter’s documentation, and indeed I did come across a section that had eluded me before.

As a colleague had not easily found this feature of JMeter either, we thought a little spreading the word could not hurt – go on reading to find out more.

JMeter TCP Sampler

By default JMeter includes a TCP Sampler that – according to its documentation

opens a TCP/IP connection to the specified server. It then sends the text, and waits for a response.

Adding a TCP Sampler to a test plan looks like this in the UI (taken from the JMeter documentation):

As you can see, there is a “Text to send” input area. This is perfectly fine when trying to talk to a server expecting clear text communications (e. g. SMTP – even though this does have a special handler included in JMeter, already), but for a binary communications protocol this is not suitable. In my case the data to be sent looks roughly like this when “forced” into text:

Obviously this cannot be put into the text field verbatim and be expected to work.

But there is a powerful configuration setting for this sampler, shyly hiding behind the simple textfield labelled “TCPClient classname”. The table describing the parameters in the documentation simply says it is optional and “Defaults to the property tcp.handler, failing that TCPClientImpl.”

TCP Client Class

There are three implementations provided for the aforementioned TCP Client class. The default one is TCPClientImpl which does what is described above: Send text and wait for an answer.

More interesting for our use case however are BinaryTCPClientImpl and its sibling LengthPrefixedBinaryTCPClientImpl.

They are described as follows:

BinaryTCPClientImpl

This implementation converts the GUI input, which must be a hex-encoded string, into binary, and performs the reverse when reading the response. When reading the response, it reads until the end of message byte, if this is defined by setting the property tcp.BinaryTCPClient.eomByte , otherwise until the end of the input stream.

LengthPrefixedBinaryTCPClientImpl

This implementation extends BinaryTCPClientImpl by prefixing the binary message data with a binary length byte. The length prefix defaults to 2 bytes. This can be changed by setting the property tcp.binarylength.prefix.length .

This is much more suitable to our needs. Encoding a binary request as a string of hex values representing the bytes is trivially easy. Either you have an editor that allows you to that – for example the very useful Hex Fiend (Mac) – or you simply drop down to the “Good ol’ command line”™:

This hex-string of the request can now be entered in the text field of the TCP Sampler configuration:

This is already quite usable, however for more than a very few requests copying the TCP Sampler entry and filling in different hex strings would not scale well. Fortunately the other JMeter features can be easily combined with the TCP Sampler as well.

External Data Sources and Variables

In my case there were several types of business transactions, each having a specific form of request, but inside the individual requests some values need to be filled in with barcodes and amounts. JMeter offers a variable replacement mechanism that can be driven by external data sources, for example CSV files.

For each iteration of a test a new line will be read from an external file and used to set JMeter variables. The mapping of the file’s columns to variable names is configurable. So first, I created a data file formatted like this:

58622199999950564FFF,000000000066
58622199999950606FFF,000000006622
58622199999950648FFF,000000001133
...

and saved it as a_transactions.csv in the same directory as the JMeter test plan.

Then I added a CSV Data Set Config element with the file name configured:

Notice that I set up the variable names to correspond to the columns in the file and also selected the appropriate options to limit this file’s data to the current thread group (more on that later) and also to “recylce” the file, i. e. start over with the first data row again when the last one has been read. This allows for continuous test runs. Of course these settings are very much specific for the test case at hand.

Finally I modified the hex dump in the TCP sampler to use the variables declared in this data set:

007d...f1f0${barcode}...${amount}...f0f2...

In the end the full test plan looked like this:

As you can see in the screen shot I defined some more global variables that are used to configure some settings in a single place. They are used to set up the several thread groups and their sub-elements. In my case, each thread group is configured to test a specific type of transaction. By tuning these global variables or by enabling/disabling single thread groups a great deal of flexibility is achieved.

Finally, running the test rewards you with a nice graph view (or whatever post processing steps you configure):

One interesting detail to know about using external files as data sources for test input is that they are not automatically sent to remote nodes, so if you need JMeter’s remote control functionality you have to make sure the CSV files are present on each node. They need not be identical in content, of course, but the file names configured in the test plan must be present on each node.

Conclusion

Even though at first it might not seem possible or easy to do, JMeter is very much suitable for binary protocol measurements, even without writing a custom plugin for the protocol at hand. This holds true, if all you need is simple roundtrip timing and if the protocol lends itself to the template based approach described above.
Of course, more complex protocols might require a more elaborate preparation and setup of the external data sources, but in many situations the out-of-the-box functionality will be sufficient.