I'm a big fan of exploratory testing. I believe it can be a much more effective method of software testing than hard-coded automation or manual scripts, both of which suffer from both inattentional blindness and the minefield problem.
The problem is explaining it. People seem to fall into two camps -
1) Those who believe exploratory testing is undisciplined hacking and will result in sloppy, inconsisent - but generally low-value results and,
2) Those who know that Sally can bring an app to it's knees in five seconds, but don't know how. This leads to the belief that ET is an attribute that certain people are simply born with, that it can't be taught, or that ET is some sort of black art that involves years of study and some sorts of secretive, back-room rituals.
I suspect the truth is somewhere in the middle; that ET is a skill that can be learned, practiced, and you can get better with it over time.
About a week ago we had this discussion on ET on the Agile Testing Email List, and I suggested that we record a simple video and put it up on youtube.
The video would show a website, give it an exploratory tester, and show what he actually does. It would probably take a few takes. First we would record what he does, then he'd watch the tape, and do the exact same thing again - but explaining his thinking process along the way.
My goal was to de-mystify exploratory testing and demonstrate immediate value - not whining that the specs are incomplete, and inconsistent, or whining that QA should have been involved earlier.
In the mean time, I went out to Google Video and checked to see if anyone else had taken this idea. While I did not find demonstrations of ET per se, I did find a Jon Bach Google test talk on this exact subject - the happy medium that treats ET as a skill, and how to get better at it. (And yes, he does a little demonstration in the middle)
Here's the link
If you read Creative Chaos, you probably don't need it - but your boss might. Or his boss might. Or your cubemate might. It doesn't get much cooler than "Well, the staff at Google HQ in MountainView thought this guy was worth listening to ..."
Heck, invite the team to a brown bag and just show the video. :-)
As for the "what an exploratory tester actually does", I think that would make a good video and am seriously considering purchasing a video camera to do it. Or, we may just record it at WOTD between sessions, I dunno.
Thursday, April 03, 2008
Subscribe to:
Post Comments (Atom)
4 comments:
Take a look at the videos from the CAST (Conference of the Association for Software Testing) 2007 Testing Challenge at www.AssociationForSoftwareTesting.org/drupal/CAST2007/Challenge . Click the team file links for videos of testing techniques demonstrated for the camera.
I highly recommend that anyone even remotely interested in testing attend CAST this year. See http://cast2008.org for more info.
Ben Simo
http://QuestioningSoftware.com
Matt,
I recorded a screen cast of me doing exploratory testing but two different software I used didn't help me to get a file out of it to upload it somewhere.
I think starting June, I am planning to come out with a series of video on it with sophisticated equipment than the ones that I used.
Thanks for posting this.
I suggested something similar to this to David Gilbert - creator of the TestExplorer. I asked him if he can creat ET sessions with likes James Bach, Jon Bach, Cem Kaner, Michael Bolton, Jon Kohl and post the videos of those sessions ... they would be great examples to demonstrate the value of ET and its relevance in todays testing.
I would love to watch James Bach doing a live ET session. Hope your efforts at WOTD will generate more interest in this area ...
Shrini
I like exploratory testing, too, although I'm not really good at it - and I would welcome any help on learning to become better.
I think you are wrong, though, that it is *more effective* than fully automated testing - it simply has a totally different goal.
Exploratory testing is *necessary* to find problems that you can or will not find by just thinking about the functionality, as well as for problems that are simply not cost effective to detect by automated testing (such as layout problems of a ui).
Automated testing, on the other hand, is *necessary* to make testing cost effective and enable rapid iteration and refactoring. If you need to run as many tests as possible every couple of minutes, there is no other solution than full automation.
So I'd argue that exploratory testing and automated testing are both needed and actually need to work hand in hand. For example, when I find a problem by exploratory testing, I will not only fix that problem, but also think about how to write an automated test so that that problem won't occur again, ever.
Which reminds of something I think Ron Jeffries once said: if you ever execute the same manual test a second time, you are wasting the tester's time and creativity.
By the way, I'm proposing a session on exploratory testing for Agile 2008: http://submissions.agile2008.org/node/276 - when it gets accepted, I hope that it will raise the awareness in the community at least a little bit.
Post a Comment