Increasing Transaction Rate: a Valid Performance Testing Technique

Just read a very good post Concurrency of Users Vs Increasing Transaction Rate by Jason Buksh about a pretty old question in load testing: if you could use less virtual users by increasing transaction rate.

Jason’s post discusses the subject in great details. One more thing worth mentioning is that the comparison between these two, say, approaches makes sense when the system handles the load well (low response times, etc.). As soon as the system starts to slow down, load becomes quite different (as far as you have a limited number of requests sent to the system with a small number of users).

Still I rather prefer to think about it not as about two equal approaches. If you use increasing transaction rate alone, it would be definitely cutting corners. Much better than nothing, but you still risk to miss issues related to high concurrency (which are quite probable).

I’d rather prefer to think about increasing transaction rate as an important performance testing technique to use in addition to using a realistic number users. This view is explained well in the Rapid Bottleneck Identification – A Better Way to do Load Testing white paper. Actually it was an Empirix white paper originally and there was a discussion about it back in 2005.

So increasing transaction rate leaves high concurrency risks and it is always better to test the full concurrency in the end. But increasing transaction rate may be a good technique to speedup performance testing and catch some issues earlier.