Along the lines of the previous post, I've just re-discovered a classic - Marick's Paper on When To Automate A Test.
Oh, it's from 1998, and it's showing it's age, but it lays down some basic dynamics in software testing and gives you tools to determine to automate, to test, or maybe not to test at all.
I think it is great /input/ to help you determine what approach to take.
But what do you think?
Sunday, July 13, 2008
Subscribe to:
Post Comments (Atom)
3 comments:
Yii! During a recent postmortem, I got in a discussion (er... ok, perhaps it was an argument) with the CTO of the company and a couple of developers who feel that "we should automate 100% of our tests". Unfortunately, I'm not always level headed during emotional interactions with highly placed individuals, so I don't believe that the details of my arguments were well presented. Additionally the room was full of disengaged Friday afternooners. I'm still livid (apparently).
So, the answer to the question "When to automate a test?" at my company seems to be ALWAYS.
Should make my task list easier to organize, but I'm starting to consider that this company may not be the place to be in an economic downturn...
So, ray, let play Dr. Phil -
Come back in six months and let me know how that's working for ya.
:-)
Seriously, I hope the link helps. I know if makes us feel good to say that all tests should be automated, but it's be tried in many context an succeeded in very few.
We need to stop talking about the way things should be and talk about what really works.
Interesting paper.
I think it's important to have a good balance between automation and manual.
Automation is great for certain applications and features. Identifying those and automating the right tasks is the difficult part.
Personally, I like the idea of having a strong set of automation tests around the engine of an application. It is easy to automate, compared to manual, and sometimes you can code a lot of examples more effectively than trying to manually test them - especially when verifying the results.
I also feel its good to have a series of smoke tests on the UI which cover maybe 10% of the features, but provide enough information to tell you if something horrible has gone wrong. The tests should be stable enough to cope with minor changes, but as they are focused on core areas maintenance is acceptable.
Combine this with the ability to execute the tests on multiple different machines and configurations and really save time and worth the additional effort.
However, it's important to know when to manually test the application - especially while doing exploratory testing to get a feel for how it works.
Combining the two is ideal. However, knowing which features to automate is always going to be difficult.
Post a Comment