Test Automation - II

I got some great comments yesterday - Charlie and Scott made some solid points, and they are points that I will address. However, before I get there, I would like to fill in a bit more of the back story from the Software-Testing List.

Here's my reply, the next day, after a small challenge by a guy named Mike Tierney:

Mike Tierney wrote:
"I would tend to agree with everything you have written or quoted Matt. But what about automated tests that are benefits for the tester ? I am not working in a TDD or extreme programming shop ... "

Two Replies:

1) I have seen test automation work well - heck, I write perl scripts that do model driven testing on a daily basis.

When I have seen system test automation work well, the tester started out with the context of the problem and built automation on top of that. Brett Pettichord's "Homebrew Test Automation", for example, struck quite a chord(*) with me. (You'll notice, for example, that Ward Cunningham's story follows that example.)

When it comes to context-free advice for a specific tool "Just use MacOSRunner and you'll be fine", I'm much more leery.

2) I like the Michael Schwern terminology for testing - I break tests into "Those the customers care about" (FIT, fitnesse, excel spreadsheets, etc.) and "Those the developers care about."

Many shops (including mine) have very technical testers, who do system testing that is more complex than what the customers care to track. They'll care enough to understand the work, maybe, but not to define or do it. I suspect that the work Mike Tierney does, if I looked at it scrupulously, would look like "Developer Tests", even though they are behavior (black-box) oriented.

But that's just me talking.

--heusser

(*) - Must ... Avoid ... Silly ... Puns ...