This morning, Shrini Kulkarni sent me an email asking what I thought of requirements based testing approaches. Specifically, his email has this line in it, quoting from a web page:
"XXYYY Inc is the world's leading expert in Requirements Based Testing (RBT). The RBT process first ensures that the specifications are correct, complete, unambiguous, and logically consistent"
Which made my spidey-sense go off. Shrini asked me to elaborate on why and what that means, and liked my reply enough to encourage me to blog about it.
Here goes:
1) On day 90 of a 150-day project, the business senses an opportunity that will allow them to double revenue. They change what the software must do. This renders the perfect spec imperfect.
2) On day 100, the VP of engineering is fired. The new VP of engineering thinks that the fribble feature is "all wrong." The spec is now imperfect.
3) On day 110, the CEO promises BIGCO that the software will do X. The spec is now imperfect.
4) On day 120, the VP of sales interprets a line item as doing X+1, and promises it to the customer.
5) On day 125, the new VP of Engineering brings back the famous consultant Rickaro BlenderBurger, to explain how he personally inspected the specifications to make sure they were unambiguous, and the X+1 promise was not right. The customer responds "But he promised us."
6) On day 126, the CEO creates a time warp field and goes back to day 120, and personally sits in the meeting with an invisibility cloak, to figure out how it is possible for the VP of sales to get the unambiguous document wrong. He brings along a linguistics professor.
The linguistics professor watches the meeting and says "Oh, yeah, that is going to happen. English was pretty much hobbled together by illiterate peasants, and codified by writers like Chaucer who knew just enough Latin to grok the letters. It is a pre-rennesaince, pre-age-of-science language. Anyone who tells you they can write an unambiguous spec in english is selling you something."
---> In other words, to quote James Bach:
"Here is a simple litmus test for an ambiguous spec: It is written in English"
Seriously. Ask an engineer the difference between and English spec and a CAD drawing.
----> People who think you can write an unambiguous spec, so you can slide it under the wall, are denying how most human beings actually communicate and collaborate.
It's not about hand-offs. It's about working together. Or, in Japanese "Gung Ho."
Hence, my spidey sense.
Regards,
--heusser
UPDATE: I am not saying that we should completely reject the idea of requirements or specs and only go by word of mouth. As with many things, the danger is in the extremes. Instead of just picking an extreme, we could have a more healthy conversation about which way we move the needle - to do a little more conversations or a little more documentations.
UPDATE TWO: Over the great back-channel of email, Jonathan Kohl points out that even though the claims of XXXYY Co. may be unrealistic, the ideas and concepts that XXXYY uses could be valuable and might be worth looking into. There are exercises that can be used to get less ambiguous documentation, and they can be helpful. As the expression goes "Don't throw the baby out with the bathwater", and I think he has a point. Thanks, Jon.
Tuesday, April 29, 2008
Subscribe to:
Post Comments (Atom)
2 comments:
>>> I am not saying that we should completely reject the idea of requirements or specs and only go by word of mouth.
If the reader of this post gets a feeling that "we" are rejecting or downplaying the role of formal, documented specifications - No we are not saying/proposing that.
I just would like to say - be sensitive to the fact that requirements (formal/documented or otherwise) are one of several *fallible* sources of information. Treat them as such. Dont take them like word from God and don't build the entire empire on this notion.
A word about human communication dynamics - Michael Bolton told me this .. as long as one or two humans communicate meaning traslate thinking into spoken words (written words) there will be ambiguity and lack of clarity.
If we happen to invent a machine like a bar code reader that read a persons mind and traslate it into a formal language ... requirements will be unclear... that is the order of the day - we have to live with it.
Structure and usage of the language like English just adds spice to this complex stuff
Shrini
Shrini said: "If we happen to invent a machine like a bar code reader that read a persons mind and traslate it into a formal language ... requirements will be unclear... that is the order of the day - we have to live with it."
Maybe, but even then things would probably be just as unclear. I am working on an open source manual testing tool at the moment. I am the designer, and my friend Aaron is the lead developer. If you were to scan my brain at any time, you would have a pretty muddy view of the requirements other than the overall vision and general ideas we are going for.
Even though I am prototyping using the same development platform he is, and he translates my screens and workflows into working code, I still have trouble expressing the requirements. I have a lot of experience with this kind of work, and can work in the IDE with the developer, communicate using the same terms he does, etc. Imagine how hard this is for business people who don't have technical skills, and haven't worked in UX and BA roles.
So far we have a short requirements document, a spreadsheet that looks like a Scrum backlog, some scribbles on story cards, and prototypes the programmer can use directly to make the app work.
For some of the trickier behaviors, we have to just use trial and error. I can't quite express what I need in clear terms. I know what behavior I am after, but I've never seen an application do exactly what I am trying to describe, so we have no frame of reference.
I check out the code, build the app, test it in a real-life scenario, and provide feedback on what feels clunky. As I use the tool, some of the fuzzy requirements get worked out, some don't. If I am using it when he isn't around, he has trouble figuring out what I'm after from bug reports. When he looks over my shoulder and listens to me complain and gesture and whiteboard, he says: "Aha!" and jumps to the IDE to work. Sometimes. Other times I need to go and alter the prototype, and show how I would use it before he gets the "Aha!" moment.
It's almost embarrassing when I try to describe my requirements as clearly as possible and struggle for words, or the right picture to convey my vision. After all, I've been working with this stuff for over a decade. I suppose this is what happens when you try to invent something new, even when it is a really simple little test tool. This exercise certainly helps me understand better what business people go through when we try to elicit requirements from them. I'll approach the next project with much more empathy when they get vague and pull out the sock puppets in a meeting. :)
-Jonathan
Post a Comment