Software Test Estimation - VII

So we've had a bit of an adventure; I've covered three different estimation models, each with a basis that isactually sound.

And there's a problem.

The problem is simple: We are presupposing that "tested" and "not tested" are binary statements. That there is some amount of testing that is "right" and it's universal. Doing less is irresponsible; doing more is waste.

I suspect there is more to the story.

---

First, consider this situation. A leader, executive, or customer hires you as a boutique tester. He ells you that "I'd like you to evaluate this product for one hour and then share with me what you know. I'll pay you a big pile of money."

In that case, I might want to be very clear about what responsibility I am taking on. If I were testing missile-launch software, I might have quite a few concerns.

For the most part, though, I don't have a problem taking on a project like this, especially if he pile of money is large enough.

In that case, an estimate is pretty easy, right? An hour, maybe two if you want a debrief and you'd like me to write up some documentation.

Most software testing projects are somewhere between "Take an hour and tell me what you find" and "how much time do you need?"

In many cases, it might be wise for us to consider test negotiation, not test estimation.

---

Test Negotiation

Management knows how long they want testing to take - they want it to be free. Or at least ridiculously cheap; they'd certainly prefer the one hour, the one day, the one week test 'phase' to whatever reality offers.

And that's okay.

So one thing we can do when it comes to testing is offer the cafeteria.

Hopefully you're familiar with the idea of cafeteria-style dining - each piece is sold separately, and you only pay for as much as you want.

The way the buffet works is that we come up with a range of plans, at a range of price points, that address a range of risks. We use the tools in the previous six posts to estimate them. Then we ask management "Which (testing) services would you like to pay for?"

Based on that discussion, we draw up a test strategy. Along the way, we make recommendations to management, but let them sign off.

This does two things for us. First of all, it transforms our roles from roadblocks and nay-sayers to consultants and enablers.

Second off all, it moves risk management decisions to the senior management layer.

I have been in meetings where someone asks "Why didn't QA find that?" and the VP of Engineering replies "I signed off on QA skipping that test; we decided to take a calculated risk to hit the deadline."

Suddenly, once the senior executive takes credit for the defect, it's not such a terrible thing that "QA missed that bug." Instead, it was a business decision.

I could get sarcastic about that, but the thing is: It's actually true. It was a calculated, explicit business decision.

Bringing our customers and stakeholders into the process will help them understand what we do, and understand the great task of "complete testing." (Whatever that means.)

It's a better place to be.

As to how to get there, and how to test under iterative and incremental models, well ... more next time.