You keep using that word ...

Last week I wrote that:

There's something going on here with the way we use terms like "Test" and "Requirement" that causes confusion and misunderstanding. Fundamentally, various groups, like the traditional test community, the "Agile" Developer Community, the Scrum People, and so on, are looking for different benefits from testing. Thus, in conversations, we "miss" each other or leave unsatisfied ... Perhaps that's a post for another day.

Then I spent the next week in a private email discussion over this subject, started by Ben Simo. During that discussion, we identified two major different kinds of tests:

A) Examples to help with the construction process. For example, in a simple Yards-To-Meters conversion program, examples can show the bounds for input, can demonstrate how many decimal places to take the rounding, what to do with garbage input, and provide samples to check the forumula. You could argue that tests as examples can act as requirements. (You Would Be Wrong, but you could argue it). Personally, I hold that these kinds of "construction/example" tests can augment requirements, and that's a real good thing.

B) Tests to learn things about the software. These tests are investigative in nature, and make sure we are not "fooled" into thinking things like "The Software Works" without reason. These investigative tests generally require much more aggressive and advanced critical thinking skills, often in real time.


---> One problem that I've seen is that we talk about these things, "tests", but don't expressly talk about which of the two classes we mean - and, of course, there are more than two classes. On the discussion list, I think Ben summed it up best:

I think we have people thinking that automated "acceptance tests" can replace traditional critical testing. We now have some testers thinking that developers are going to be more involved in the critical testing. People coming from the two viewpoints don't seem to realize that they aren't all talking about the same thing. ... Although there are some testing ideas that are not compatible, I do believe that things like TDD and automated acceptance tests are good as long as people don't think that automated execution replaces restless thinkers. If I had to have one or the other, I want the restless thinkers. However, I hope we can have both.
- Simo

---> Now, for my thoughts. I've spent most of my career with a business card that said "Developer." When I started doing programmer-testing, I did actual critical thinking and "does it work" testing.

Then I ran into various Agile dev/testers that, well, weren't. They were using tests as examples and not finding bugs - or writing blogs, articles, or books about "testing" that didn't really talk about critical thinking.

My initial response to this was:

"What the hell is wrong with you people?"

After some time, I realized that a different philosophy about software testing leads to different results.

If those are the results you actually want, well ... I guess that's ok.

In the free market of ideas, these ideas will compete - and sometimes, complement each other.

May the system that delivers that best result win.