Good Times for Load Testing

It is an interesting time for load testing. Neotys just announced a new Neoload release – and its full-functional Free Edition (up to 50 virtual users). Just after HP announced LoadRunner free Community Edition , also up to 50 virtual users. And the first move was probably done by SOASTA with CloudTest Lite (up to 100 users – but with some quirks like VM-based distribution).

We also see premium services based on open source product, like provided by Blazemeter on the top of JMeter, bringing open source products to enterprise level and, in a way, reinvigorating them. The combination that in some sense boils down to the similar model – free product for small-scale and simple tests, premium service for large-scale tests – but without limitation of virtual users for free option.

That looks like a major change in the market – for a long time open source and free initiatives were rather marginal in load testing. While there were many open source products – but just few of them were worthy to looks at. Except JMeter, maybe Grinder was noticeable at some point. Attempts to spin off open source projects, such as OpenSTA from Cyrano code or WebLOAD Open Source Load Generation Engine sponsored by RadView, basically failed. And now we have commercial vendors moving to the freemium model and see some revival of open source tools (which may be a reason behind vendor policies change)– with JMeter and Gatling probably most visible.

So it looks like it becomes an industry standard – and you don’t need to pay for tools for low-scale tests anymore. That should have a very profound impact on the industry. It is probably difficult even realize all possible consequences for the moment – but we haven’t seen such market changes in load testing for ages.

First, it comes in time for the latest industry trends. With agile development and continuous integration (CI) / delivery / deployment the old approach when only a dedicated performance tester works with the tool becomes a history. You need to have an opportunity for developers and other testers to use tools, invoke tools automatically as part of CI, etc. It probably won’t be large-scale tests anyway – 50 users should be enough for most of such tests. So you get it covered for free – and need to pay only for large-scale tests (often as needed).

Second, it may be a serious blow to low-cost and free products that doesn’t provide real value comparing to market leaders. It may lead to some market consolidation, leaving only the fittest.

Third, it opens better opportunities for people to get familiar with the leading products and get some experience with them. Probably it should attract more attention to load testing, lead to creation of better information infrastructure (blog posts, books, forums, etc.).

Fourth, new premium solutions based on open source products (such as Blazemeter) should invigorate open source market (as far as such service providers, benefiting from the products, will invest back into them to bring them up to their needs – and perhaps we will see something more similar with other areas where open source software is blooming based on similar business models).

I am probably missing some potential effects – but even listed above should be enough to realize that we are seeing drastic changes in the load testing tool market – which was stagnant for a very long time.


Leave a Reply

Your email address will not be published. Required fields are marked *