We use the open source PerfHarness tool with point-to-point messaging over HTTP/TCP or MQ/TCP to evaluate the performance of the software systems under test. Regardless of the transport protocol used, a number of PerfHarness client threads simultaneously send prefabricated payload messages to the system under test, wait for and get a response, and keep repeating this synchronous request-response interaction with the same payload for the duration of the test. We refer to the collective of all request-response interactions that have taken place during this predefined time interval as a test run.
In accordance with the above, we use HTTP synchronously and do not pipeline requests in a connection. The connections are kept open for a predefined number of request-response interactions as allowed by the HTTP persistent connections scheme. Once the predefined number is reached, the connection is closed and a new one is opened in its place. This process is repeated on each connection until the predefined length of the test runs out, at which point all connections are closed regardless of how many requests have been sent over them, but not before a reply has been received for the request currently in flight over the connection if there is one — and there can only be at most one as we don’t pipeline requests.
When using MQ, PerfHarness client threads put prefabricated MQ payload messages to a queue designated as an input to the test case, synchronously wait for and get a response from the designated output queue, and keep repeating the put-get cycle for the duration of the test. As is the case with HTTP, we wait for a response for all in-flight requests before ending a test run. Some tests generate messages on other queues in addition to the output queue; in such cases we use additional PerfHarness clients to remove messages from these extraneous queues to avoid them filling up and blocking the test run.