And now I continue my post Can Processes Be Agile? into its application to testing (I broke it into two to avoid complete mess).
In a way I think that Exploratory Testing is an application of pure Agile principles to testing. And it looks like adherence to pure agile principles makes it rather difficult to explain – I guess that What Exploratory Testing is not series is an evidence how had is to apply pure agile principles without bringing it down to just another described in details process.
I have a small stake here: in 2008 I wrote a paper titled Agile Performance Testing. I shared a few ideas on how to do performance testing in a more agile way there (using the word agile according to my reading of Agile Manifesto). By the way, I never meant that it is a new way of performance testing – rather a collection of several practices that, I believe, good performance engineers use (consciously or unconsciously). The paper didn’t touched at all performance testing in agile software development, so some attendees were disappointed (well, in its latest reincarnation I presented at Agile Testing Days I added slide 47 about it). Should admit that the title was chosen to provoke interest to the topic and it fulfilled this purpose well. Still I was periodically wondered if the name was correct – and still think that it does.
Moreover, considering that Exploratory Testing is, in my current understanding, is application of agile principles to testing, I wonder if I should name the next reincarnation of this presentation “Exploratory Performance Testing”. Indeed the point I am trying to convey is that simultaneous learning, test design and test execution is even more important in performance testing. In functional testing you may have something (like use cases) describing how the system should work, but you don’t have anything that would describe how the system would behave from the performance and resource utilization point of view (performance requirements saying what are the limits stakeholders wants the system to achieve, but that’s the best you can hope to get).
At the CMG’12 conference we had an open discussion “New Approaches to Performance Testing: An Open Discussion on Plans, Experiments and Points In-Between”. The point I was trying to convey was that you need to experiment in performance testing to learn how the system behaves; just following a formal plan is less effective and often leads to missing performance problems altogether or to prolonged agony of performance troubleshooting. I, of course, mean a system of experiments based on the information we have about the system and results we are getting – it looks like some people believe that to experiment means to do random things. Well, it looks like I wasn’t too successful in this. Probably I should learn more how people advocate Exploratory Testing – but it looks like it is not easy to advocate something that doesn’t look a straightforward process.