Does anybody else remember the Middle School Science Fair? The idea is to apply the scientific method to something practical, then show off all the budding scientists.
If I remember the scientific methods as I was taught, it was essentially this:
- Collect some documented observations of your subject
- Make a hypothesis – a theory that should predict behavior (Make sure you pick the right one!)
- Conduct an experiment – Conduct it properly, so to easily repeatable.
- Evaluate the results of the experiment – do they support your hypothesis?
- Draw Conclusions
When done, you should have some artifacts that document the experiment. This is usually a big poster-board with a page for each step in the method, plus a cutesey visual display. We were then evaluated on our displays, documentation, and how valid our experiments were.
... and I suspect that the entire approach is wrong. Wrong. Wrong.
What if we were not taught to do one experiment meticulously and correctly, but instead taught how to do lots of experiments – dozens, hundreds – very quickly and very sloppily? Then, if we find something interesting, we can come back and do the experiment again. If the results are interesting, then play the documentation game.
Done this way, our students actually have a shot at real innovation, instead of pseudo-science. Thus we can judge competitors not by the quality of the documentation, but by weather or not they actually found anything interesting.
This test early, test often, test always idea of science is not new; actually, it’s the basic idea that Thomas Edison used to bring the light bulb to market. In fact, when I looked up the Scientifc Method in Wikipedia this morning, it had a large section on "Evaluation and Iteration" and "Testing and Improvement." It turns out, that’s the real scientific method after all.
The problem is that we are so seeped in this culture that we confuse the accidental things like documentation with the essence – discovery and experiment.
We have this problem in software testing.
We call it the test case.
Schedule and Events
March 26-29, 2012, Software Test Professionals Conference, New Orleans
July, 14-15, 2012 - Test Coach Camp, San Jose, California
July, 16-18, 2012 - Conference for the Association for Software Testing (CAST 2012), San Jose, California
August 2012+ - At Liberty; available. Contact me by email: Matt.Heusser@gmail.com