Test Estimation - IV

So far, I've pointed out various issues in test estimation, pointing out the fallacies involved in simple 'naive' test estimation. I also pointed out that is possible to do some kind of estimate, even if all you say is "um, yeah boss -- two weeks."

A couple of people expressed surprise at this. Laurent Bossavit, a board member of the Agile Alliance, was concerend that people wouldn't take me seriously.

But if you've been around the block a few times, you might just remember a test project or two where a wild guess made without context was the test strategy. Someone with no context walked in, said "testing -- hmm. Four weeks." and walked away. They were the boss, and it is what it is, and the team made the best of it.

Hopefully that isn't what actually happened.

It might have looked like that to an outsider, but there was likely something more going on. Perhaps the manager needed to ship the product in four weeks in order to hit some deadline. Perhaps he needed the staff to stop billing on project X because in four weeks, you'd run out of budget.

Or, perhaps, it's possible that some sort of rational process was going on in his head that we could not see.

I'm going to start with the rational approaches. Don't worry, we'll cover the social approaches, and even agile and iterative models -- but I think the next step is to have some logical reason to believe it will take X, instead of wishful thinking.

---

It might have looked like "Four weeks" was made up, but it's very possible that some complex process was going on in that manager's head. For example, he might have thought:

Hey, this project is about as big as project A. I've got the exact same technical teach as project A. We haven't had much turnover in the dev ranks either. We've actually learning something since project A, and I believe the team will make less mistakes. How long did project A take to test? Oh yeah, four weeks. Okay. I'll use that as estimate.

Check it out -- our theoretical manager might actually had a reason for believing in the four week number.

It's also possible that the manager was considering a number of projects and averaging the lengths, to come up with four weeks. It's unlikely the data was available, but the human mind is surprisingly capable of doing those averages, even subconsciously. I prefer to do it with a pen and paper and a little math, to have something to explain to someone who asks "where did the four week number came from?", but many a tester, lead, or manager can do this sort of comparison in their head without even realizing it.

This happens in other disciplines too. Consider the expert Butch who has spent his entire adult life in the field. You ask him for two pounds of ham and he goes slice slice slice - weight it and it's exactly 2.002 pounds. Ask him how he did it, and he'll likely say "I don't know. I suspect cutting meat five days a week for thirty years had something to do with it.

---

But we can do one better than that. Write down a half-dozen projects. It's unlikely that any of them are specifically like project X. Project A had a different dev team, project B had more testers, project C was riskier with a more complex infrastructure, project D was the first time we had ever used the new webserver, and so on.

So you end up with a range. "Given our previous projects, we expect this test project to take three to five weeks."

Of course, there's all kinds of reasons that's a bad estimate. But it's better than the first strategy - to make something up - right? And it's likely better than functional decomposition, for reasons we already discussed.

This idea - past history - is a lever, a way to come up with an estimate. In other words, it's a model from which an estimate will "pop out".

When this series is done, you should have several levers. You'll be able to apply several models to your project, get several estimates, and consider which one is correct.

More to come.