One way to transition to agile

There's been a little bit of discussion in the blog-o-sphere lately about how to "sell" agile, or how to convince senior managers to "adopt" agile, how to get buy-in and so on.

I pushed back against this; couldn't you just do it? Does Senior Management even know what the software team is doing, anyway?

The answer to that was that Agile is an investment; a typical team might take six to eight months to build infrastructure (CI, Test Driven Development, deployment tools, etc) and culture -- eight months before the team is again productive. (Reference)

Now there are a couple of different ways to transition to Agile; a team might, for example, make a series of small, incremental changes, each of which pays for itself quickly. But I thought it might be nice to share my favorite "sell the CFO" transition to Agile Story:

A long time ago ...

Senior Manager: "We need you to be the technical project manager on the Super-wiz ERP upgrade, Dave, so we can sell (new product) by (date). If we aren't int he market by (date) (competitor) will eat our lunch. Due to government regulation, we need to file a plan by (date1) and start selling on (date2), or we miss a ONE YEAR market window, by which competitor will have sewn up the market. Dave, you are the man. Only you can do it."

Dave: "I need a war room and for the entire team to be physically co-located, 100% of the time."

Senior Manager: "Well, I don't know about that."

Dave: "If you want me to have a chance to hit your date, I need a war room."

Senior Manager: "I'm sure you can do it. We have confidence in you."

Dave: "If you want me to have a chance to hit your date, I need a war room."

Senior Manager: "Dave, politically speaking, it's impossible. I could probably get you all the technical folks in one room, but then we'd have to find the room. No, you'll have to make it work.

Dave: "If you can't find a war room, then I'm not the person to manage this project. Perhaps you can find someone else willing to take that problem on. I am not."

Senior Manager: "But you're the best! There is no one else who can do this."

Dave: "I need a war room."

Dave got his war room.

The Moral of the Story

One time that organizations are willing to drop "the way we always do it" and try something new is immediately before an oncoming crisis. If you step into the void and offer to take responsibility given certain reasonable changes, you've got a real shot at impacting long-term change.

Another way to get chance is to deal with an organization that is profitable enough that they can experiment.

The classic example of the intersection of those two problems is the creation of the IBM Personal Computer.

The Scrum literature is full of examples of this sort of game-changing project; the classic example is probably in "Wicked Programs, Righteous Solutions" by DeGrace and Stahl.

The Drake Equation of Software Testing

In the 1960's a scientist named Frank Drake came up with a formula to predict the probability of life on other planets -- specifically the chance they would evolve to the point that we could contact them. That equation came to be known as the Drake Equation.

The drake equation is roughly this:

N = R * f(p) * n(e) * f(e) * f(l) * f(i) * f(c) * L

Where:

N = the number of civilizations in our galaxy with which communication might be possible;
R = the average rate of star formation per year in our galaxy
f(p) = the fraction of those stars that have planets
n(e) = the average number of planets that can potentially support life per star that has planets
f(l) = the fraction of the above that actually go on to develop life at some point
f(i) = the fraction of the above that actually go on to develop intelligent life
f(c) = the fraction of civilizations that develop a technology that releases detectable signs of their existence into space
L = the length of time such civilizations release detectable signals into space.

This sounds impressive. I mean, if we could just determine those other variables, we can determine the chance of life on other planets, right?

But ... Wait

It turns out that Drake's equation really says that one unknown number that can only be guess can be calculated as a function of seven numbers ... that we don't really know either and can only be guessed. (And if you try, you run into Kwilinski's law: "Numbers that are 'proven' by multiplying and dividing a bunch of guesses together are worthless.")

Now think about this: As an actual menaingful number, drake's formula is pretty useless. If any of the guesstimates are off by a wide margin (or you can't even really predict them at all), then your answer is a non-answer. Worse than no answer, a wrong answer wastes your time and pushes you toward bad decisions.

Yet what if we didn't try to come up with a 'solid' number, but instead use Drake's equation as a modeling tool -- to help us better understand the problem? To help us figure out what questions to ask? To guide our research?

Suddenly, Drake's equation has some merit.

Back to software testing

Over the next few months I plan on doing some work in the area of the economics of software development -- testing specifically, but also other aspects. To do that work, I intend to throw out some illustrative numbers to help model the problem.

I don't claim that those numbers are "right", nor that any numbers are "right"; the economic value of the software project will develop on what the project is, who the customer is, what the technology stack is, the value of the staff, the time in history ... illustrative numbers are overly simplistic.

So I'm going to abstain from "proving" final answers using those numbers, instead using them as illustrative numbers, to tell as story - that it could be conceptually possible for a certain technique to work if things turned out like the example.

With that foundation model in place, we can make different decisions about how we do our work, and see how it impacts the model.

I want to be very clear here: I'm going to throw up ideas early in order to get feedback early. The ideas I throw out may be cutting edge -- they will certainly be wrong, because all models are wrong. But they might just have a chance to positively impact our field.

I figured it's worth a try.

Plenty more to come, both here and on the STP Test Community Blog.

Types of testers and the future - I

Ok, ok, it's not literally called boutique tester. But listen to this:

You: someone who can make guesses about how a website or application is going to fail, prove that you're right, and then communicate it clearly and effectively to the folks who need to fix it. Ideally, you also have the ability to predict the things about our products that will confuse or dismay a new customer.

As an example of what we do, imagine we have four months to write, test, and ship two applications on a brand-new hardware platform (with no specimens of said hardware in the building) while still updating and maintaining our released products on both platforms. Could you keep up without going totally crazy in the process?

If so, we'd like to hear from you — we're looking to expand our QA department by adding another Software Test Pilot.

Here's the ad, along with the companies "Jobs" site. The position is in Seattle, Washington, and I suspect the position is everything it claims to be.

About the future of software testing

For the past couple of year I have been engaged in a sort of shadow-boxing match of ideologies with some folks. Predominantly in the United States it has been with the idea of the tester/developer - that "the tester of the future" /will/ be writing test automation code, not doing actually testing, and the customer-facing tester will "go away."

Now certainly, I grant that testers will become 'more' technical over the next decade or so, but then again, our entire society is becoming more technical.

I still think it will take a variety of skill sets to find defects on different types of projects. We might have more tester/dev-ers in certain places, but I think "going away" is a bit strong.

In fact, I believe there are some simple economic conditions that make a boutique tester a valid choice for a company like "the omni group."

More to come. Or maybe more on test estimation to come. Either way, I'm writin' again. :-)

CAST 2011 - August 8, 2011, Seattle Washington

If you've been waiting with breathless anticipation for announcements on CAST 2011, wait no more: It's August 8-10 at the Lynwood Convention Center in Seattle, Washington.

Note: I am not running the conference, and this is not some sort of official announcement. I have been, however, monitoring the intarwebs closely, waiting for such an announcement, and wanted to be the first to link to it once it came out. :-)

You heard it here first, folks.

Alternatively, maybe you didn't, in which case, I admire your testing-web-search-foo.

So perhaps I should say "you probably heard it here first?"

I am unaware of a Call for Proposals. As soon as I know of a public one, I'll link to it.

See you in Seattle in about nine months?

Quality is a slogan

Some long-time readers may recall that I am a member of the American Society for Quality, one of the larger, older and more influential non-profits in the Quality Space. ASQ has historically focused on manufacturing quality, but ASQ does have a journal and certification program on Software Quality.

This year ASQ is starting a new initiative on "raising the voice of quality", including blog posts from it's executive director, Paul Borawski.

As someone active in the quality community on the intarwebs, the ASQ leadership asked me to participate in a program called "Influential Voices." As much as I am enjoying the test estimations series (with more to come), I hope you can agree this opportunity was too good to pass by.

The first assignment was to write a response to Mr. Borawski's first blog post. Here goes ...

---

On the ASQ Blog, Paul Borawski wrote:
"Here’s my conundrum. Over the past 25 years I’ve heard some of the world’s best leaders extol the virtues of quality to improve bottom lines, top lines, and whole organizations. I’ve been in rooms of thousands when hospital executives, school superintendents, and city mangers all tell their stories of discovery, learning, trial and success. They tell their stories of transformation from the edge of failure to the summit of success. In the ‘80s I listened to Roger Milliken say, if quality is right, everything else follows. In 2010 I heard Alan Mulally (CEO, Ford Motor Company) in essence say the same thing. I get inspired and excited. Quality really does offer answers to what organizations most need.

So, what would it take to get the world’s attention to focus on that truth? What would it take to have the world realize the full potential of quality?"

---

There's certainly a lot in there, but let's start with this idea that many big, important people have championed the value of quality.

Well, um, of course they have, right?

I mean, can you imagine the CEO of Ford stepping up and saying "Quality? Nah, not really. I mean, that's not /really/ important."

I have a hard time imagining any corporate executive saying that. After all, Quality is sort of like Mom, Apple Pie, or "The Children Are our Future", the kind of thing you can appeal to and get easy, universal agreement.

Why, right now, I'm looking at the side of a five-dollar pizza box, from a company known to use big vats of pre-processed cheese, pre-processed tomato sauce, and pre-cooked, frozen pizza crusts. (In other words, the cheapest pizza with the simplest assembly-line process.)

What does the side of the box say?

---

Our QUALITY! QUALITY! Pledge
(company name) uses only the finest ingredients to provide top-quality products

---

No, seriously, that's what it says.

Clearly, something is going on here.

I'm all for quality ... as long as it doesn't cost me anything

That is to say, Americans want quality, but they don't want to pay for it.

Maybe it started with Phil Crosby's little book, Quality Is Free, in 1979. The basic theme of Crosby's work was sound: If you do a better job, you'll have less defects to fix and correct later, and defects slow you down. So by investing in quality, we actually get payback, whereas trying to go fast right now actually slows things down in the long term.

The problem is, Crosby's book wasn't called "Quality is an investment", it was called "Quality is free."

I doubt I am the first person to observe that our culture wants quality -- as long as they don't have to pay for it.

Quality is ... wait, what is quality, again?

Let's be honest. We don't agree on what quality is. Joseph Juran told us quality is is fitness for purpose, Crosby says it is conforming to requirements, while Jerry Weinberg suggest that quality is value to some person. (I'm with Jerry, for the record.)

That lack of definition introduces some problems. For one thing, it brings us people counting defects, arguing a lower number of defects is a more "quality" piece of software.

But there is another, more sinister problem: If you've never defined what quality is, yet agree it is good, then we've got a situation. This "desire" for "that thing, you know, that quality stuff", creates a market opportunity for consultants, gurus, authors, and all kinds of leaders.

That's not necessarily bad. I'm a guru of sorts, and, on a good day, I have had a few people tell me they have succeeded with my ideas, which is very gratifying. But a market without discernment ... that troubles me.

And it gets worse

To most folks in quality, the "Japanese revolution" is old news, so I will be brief. Over a period of a few years, the Japanese changed their reputation from making cheap plastic junk to making high-efficient, low-defect, last-a-long-time electronics and automobiles.

They went from followers to leaders.

About that time (surprise surprise), back in the United States, we developed a market for "that quality stuffs", but, as the secret reply also went "...keep in mind, we're busy with the business of production. So let's hire some bright young engineer and give him the role of director of QA."

Another common secret statement was "But don't change the way we do business, or interrupt production!"

Can you imagine why that didn't work?

Shocking.

Another monkey-wrench in the works

About a hundred years ago, a man named Frederick W. Taylor wrote a book entitled "The Principles of Scientific Management"

I would be reluctant to call the book science, by any stretch of the term, but it is worth reading, if only to know your enemy.

Taylor was kind of jerk.

His basic idea was to "separate the worker from the work" by having an elite group, the scientists, design the work to be done a specific, exacting way.

Another term for this is "defined process."

And, while Taylor's "Scientific Management" might have worked in 1910 to move iron from one place to another (though modern historians debate that), it certainly doesn't work well to create intellectual property, like books or software.

Imagine a publisher telling an author or artist when to sit down, when to write, and when to take breaks. It just doesn't work that way.

My research into the Japenese Revolution, specifically Toyota, shows a style that is roughly the opposite of Taylor. The Toyota way involves engaging the workers in the work by giving the team interesting assembly problems and asking them to find the best way to build the product.

For that matter, forget "best"; the Toyota way asks the team to continually improve the process of building the product.

So we Americans sent people over to Japan to figure out what they were doing and systematize it, just as Ray Croc had done with McDonalds.

So we came back with Lean and Six Sigma and standardization.

But something happened along the way. By translating these ideas from the far-east, we injected some of that old "Tayloristic" ideology. America, fresh from the great success of McDonalds, was in love with systems and process and standardization. We were also in love with command and control methods of management.

Nobody seemed to recall that McDonalds had to experiment first, to find a good burger that could be made quickly; we just thought standardization was good.

Now on Toyota's teams, it is the team that decides what things are worth writing up standards for, to help the new guys. On a Toyota team, making a standard pre-maturely is a kind of waste; it causes you to make rules today that will be out of date tomorrow. Or worse, causes you to build things in an old, out-of-date, slow, error-prone way because "that is what the process book says."

Yet I've read several books by Americans on "Lean Manufacturing", and can not recall ever reading anything about just-in-time standards. Instead I read "standardize, standardize, standardize"

Why, we even have a "quality system", ISO-9001, that is all about documenting processes.

About ISO-9001

The basic belief system of ISO-9001 seems to be: If you do something, you should do it the same way, every time. You should write down how you do things. Then you can hire people to come into your shop and audit, to see if you are in fact doing things the way the book says.

All of this pre-supposes that it is possible to write down a process that will lead to zero defects. With hardware, where you can "insert tab A into slot A" in a cookie-cutter process, that might be possible. In software, where the process has to say, at some point "... and then the developer writes the code", or "... and then the tester does some testing", it'll be impossible to have that kind of certainty with defined process.

Also, this pre-supposes that having new ideas, and changing our process, in the moment, is less valuable than having a stable, predictable, repeatable process.

And that, since the process doesn't change too much, the cost of documenting and maintaining change control over the process will not be overly onerous.

System Effects

So we've got a system where we define "quality" as good and no further, which creates a market demand for a product that is hard to evaluate. We have these leftover ideas of class warfare from the 19th century that are clouding our vision. We have traditional command-and-control managers that want "that quality stuffs" as long as that doesn't mean any change to them or their power base.

So people who come in selling "quality" as something that doesn't require change, that reinforces command and control, will likely have more success than those trying to reform things.

To go back to Mr. Borawski's charge, I have to say, the question should not be "Why hasn't quality had more impact?" but instead, we should celebrate the little impact we have had.

My fix? Well, I actually have some good news

The Good News

Outside of the government and some elements of the health care sector, most of North America is living in a free market economy. That means we get to compete against other companies for products and services.

If Quality is value to some person, as Weinberg tells us, and the customer is that person, then those of us who build better products will sell more of them, make more money, and have successful businesses.

Of course there are exceptions; big brands getting by on reputation, companies creating monopolies or manipulating the market. But, by and large, those focused on the "accidental" side of quality - meaningless metrics or silly process - will not do as well as those focused on excellence in product development, operations, or sales.

In other words, in the long term, the good guys win.

Now as for the other concern -- that this idea of doing quality work can lead to better products, more products for less cost, and lower defects -- well, I agree. If you think about it, "if everybody did it", we would create an abundance of resources (by which I mean, actual stuff) and make the world a better place.

Where can we start? Well, once place to start, I think, would be by giving examples and case studies of successful organizations, along with exploring ideas behind systems thinking.

Sure, Six Sigma and ISO-9001 might be tools, and they might have a place, but they are relatively shallow tools with a niche place. Let's radically de-emphasize the role of those tools, choosing instead to focus on systems thinking.

To do that, we would first model our workplace as a system, then see what the right tool is for the job, focusing all the while on the customer and customer needs.

Of course that's only a start. We'll have a whole year to debate these ideas back and forth, and I suspect we'll end with something strong than that.

But it is a start.

Now get out there and change the world!

On Skill

I just got back from the software test and performance conference in beautiful Las Vegas, Nevada - conference write-up here.

Just after I got back from the conference, we started talking about skill on the software-testing discussion list, and I posted this:

---

--- In software-testing@yahoogroups.com, "Elena" wrote:
>
> The rest I cannot quite explain to myself -- is it intuition,
>having a bigger picture/snapshots of what it's supposed to be
>like in my head, etc?
>

There's an old saw that an apprentice butcher is sitting with an old butcher.
You know, they grey-hair type who's working in the shop hit entire life.

Customer comes in "I would like two pounds of pork."

Chop Chop Chop. "Here you go."

Suddenly the kid pipes up "No way that is /exactly/ two pounds! You didn't even
weigh it!!"

"Ok, kid, you weigh it then."

Sure enough, 2.01 pounds on the first slice.

"How did you /*do*/ that?"

"I'm not exactly sure, kid, but, now that you mention it ... I suspect cutting
meat five days a week for thirty years had something to do with it."

Moral: Don't let other people label your hard-won experience as "intuition" or
"talent."

No, I suspect you /earned/ and /developed/ skill over time.

Skill takes work.

To be precise, the human brain has a lot of functions, one of which is a really
big neural network. So we recognize patterns.

You recognized a pattern that no one else could. That's part of what good
testers do.

I'm happy for you, man.

Don't let the anti-skill ... goofballs get you down.

all my best,


--heusser

---

Then this afternoon Justin Dessonville posted this video to his twitter account.

It is a five-minute video of people doing the seemingly impossible: Multiple far-court free-throws in a row in basketball, a guy who throws a playing card and uses it to blow out a candle - multiple story jumps, tony hawk's impressive skateboarding feats.

I'm not sure how some of those were done; the guy might have spent six hours in front of the video camera to record one take. But some of them, like the basketball player and Tony Hawk, were clearly the result of a lifetime of practice.

Now compare that to your typical software ideology of having a defined, standard, predictable, repeatable process.

How do you write down a process called "be an awesome skate-boarder?"

You don't.

You can, however, defined moves and create a place to practice consciously. You can memorize, and repeat, and build from low-skill to high skill exercises.

I'm pleased to say that, at this point, I see a substantive part of the test-o-sphere moving from these shallow notions of repeatability into something more meaningful. If it's testing dojos or weekend testers or something else, we're finally getting there.

I'm pleased.

Now, if you'll excuse me, I gotta go practice my javascript blackflips on IE6 ...

Software Test Estimation - VII

So we've had a bit of an adventure; I've covered three different estimation models, each with a basis that isactually sound.

And there's a problem.

The problem is simple: We are presupposing that "tested" and "not tested" are binary statements. That there is some amount of testing that is "right" and it's universal. Doing less is irresponsible; doing more is waste.

I suspect there is more to the story.

---

First, consider this situation. A leader, executive, or customer hires you as a boutique tester. He ells you that "I'd like you to evaluate this product for one hour and then share with me what you know. I'll pay you a big pile of money."

In that case, I might want to be very clear about what responsibility I am taking on. If I were testing missile-launch software, I might have quite a few concerns.

For the most part, though, I don't have a problem taking on a project like this, especially if he pile of money is large enough.

In that case, an estimate is pretty easy, right? An hour, maybe two if you want a debrief and you'd like me to write up some documentation.

Most software testing projects are somewhere between "Take an hour and tell me what you find" and "how much time do you need?"

In many cases, it might be wise for us to consider test negotiation, not test estimation.

---

Test Negotiation

Management knows how long they want testing to take - they want it to be free. Or at least ridiculously cheap; they'd certainly prefer the one hour, the one day, the one week test 'phase' to whatever reality offers.

And that's okay.

So one thing we can do when it comes to testing is offer the cafeteria.

Hopefully you're familiar with the idea of cafeteria-style dining - each piece is sold separately, and you only pay for as much as you want.

The way the buffet works is that we come up with a range of plans, at a range of price points, that address a range of risks. We use the tools in the previous six posts to estimate them. Then we ask management "Which (testing) services would you like to pay for?"

Based on that discussion, we draw up a test strategy. Along the way, we make recommendations to management, but let them sign off.

This does two things for us. First of all, it transforms our roles from roadblocks and nay-sayers to consultants and enablers.

Second off all, it moves risk management decisions to the senior management layer.

I have been in meetings where someone asks "Why didn't QA find that?" and the VP of Engineering replies "I signed off on QA skipping that test; we decided to take a calculated risk to hit the deadline."

Suddenly, once the senior executive takes credit for the defect, it's not such a terrible thing that "QA missed that bug." Instead, it was a business decision.

I could get sarcastic about that, but the thing is: It's actually true. It was a calculated, explicit business decision.

Bringing our customers and stakeholders into the process will help them understand what we do, and understand the great task of "complete testing." (Whatever that means.)

It's a better place to be.

As to how to get there, and how to test under iterative and incremental models, well ... more next time.

Interlude - About Software Engineering

(More Test Estimation to come, but first: An Interlude)

By now many of us know the standard 'barbs' - for example, that when Winston Royce designed the waterfall model, he said it was "risky" and "invites failure".

Or that Tom DeMarco, author of the oft-quoted "Controlling Software Projects: Management, Measurement, and Estimation" has essentially recanted his claim that "You can't control what you can't measure."

(To be more accurate, DeMarco actually argues that the quote may hold. It just turns out that control isn't all that important on software projects these days.)

Then I saw this video by Glenn Vanderburg at the Lone Star Ruby Conference:



Once again someone has taken some of those ideas, deconstructed them, and re-packaged them in a way greater than the sum of their parts. In the video, Greg goes way back, explaining not only what Winston Royce wrote, but the how and why it could have been perverted in the "waterfall" "standards based" and "software engineering" approaches of the 1980's and 1990's -- and what we should do about it.

I think you'll enjoy it.

Next time: Still more test estimation to come.

We're getting there. Really.

Test Estimation - VI

So far, we have two ways to predict project outcome:

First by comparing the test effort to other projects, and suggesting it is "between the six of project X and projec Y", this giving us a range.

Second by calculating out test costs as a percentage of the development (or total project) effort, looking at the official schedule and projecting out our expected project length. If we're smart, we also take into account slips to get that percentage.

A third approach I can suggest is to predict the cycle time - time to run through all the testing once. I find that teams are often good at predicting cycle time. The problem is they predict that everything will go right.

It turns out that things don't go right.

Team members find defects. That means they have to stop, reproduce the issue, document the issue, and start over -- that takes time. More than that, it take mental brain energy; the tester has to "switch gears." Plus, each defect found means a defect that needs to be verified at some later point. Some large percentage of defects require conversation, triage, and additional mental effort.

Then there is the inevitable waiting for the build, waiting for the environment, the "one more thing I forgot."

So each cycle time should be larger than ideal - perhaps by 30 to 40%.

Then we need to predict the number of cycles based on previous projects. Four is usually a reasonable number to start with -- of course, it depends if "code complete" means the code is actually complete or not. If "code complete" means "the first chunks of code big enough to hand to test are done", you'll need more cycles.

If you start to hear rhetoric about "making it up later" or "the specs tooks longer than we expected, but now that they are solid development should go faster", you'll need more cycles.

(Hint: When folks plan to make it up later, that means the software is more complex, probably buggier, than the team expected. That means it'll take more time to test than you'd hoped, not less.)

---

So now we have three different methods to come up with estimates. With these three measures we can do something called triangulation - where we average the three. (Or average the ranges, if you came up with ranges.)

When that happens, it's human nature to tend to throw out the outliers - the weird numbers that are too big or too small.

I don't recommend that. Instead, ask why the outliers are big or small. "What's up with that?"

Only throw out the outlier if you can easily figure out why it is conceptually invalid. Otherwise, listen to the outlier.

Which brings up a problem -- all the estimating techniques I've listed so far have a couple of major conceptual flaws. And I haven't talked about iterative or incremental models yet.

They are just a start.

Still more to come.

Test Estimation - V

So one way to estimate the testing phase (if you have such a thing), or at least testing activities, is to compare the test effort to the development effort or overall effort on other projects.

Examples:

"We spent about ten solid months on MaxStudio, and only spent two months testing. So I think testing should be about 20% of the overall dev budget."

"We spent a year on StudioViz from kick-off meeting to release, and about a month testing. So I think testing should be less than 10% of overall budget."

---

Both of these examples are real.

The thing is, after release, we spent the next six months fixing MaxStudio, and took a serious hit in the marketplace or reputation.

Likewise, we developed StudioViz incrementally, with many stops along the way to bring the work-in-progress up to production quality. StudioViz was also a browser-based application - well - sort of. It ran in browser control inside a windows application. So we were able to 'lock down' the browser to at least modern versions of Internet Explorer.

What all this means is that if you pick history to do a percentage of effort measurement, make sure the development model - the "way you are working" is relatively similar. Big changes in teams, in technology, technique, or method can render these sort of projections obsolete pretty easily.

But now we have two methods: Comparing test effort to test effort on similar sized projects, and using test effort as a percentage of dev effort. (That is, what percentage was it of dev effort for previous projects, look at dev effort for this project, multiply by percentage, get test effort.)

Of course, both of those measurements assume that you have roughly the same portion of developers to testers - but like I said, changing things makes projections based on past experience less and less accurate.

Of, and successful organizations tend to change. A lot.

Another method, that I've hinted at, is percentage of the overall project. Now for this to work you have to be careful, because it's very hard to measure the effort if you go back to when the idea was a gleam in someone's eye. When I've done this, I've tried to go back to when the initial kick-off happened - at which point the project probably had a full-time project manager, 'assigned' technical staff, maybe a business analyst.

---

Here's another quick exercise for the folks that complain "that sounds great, but we don't have the data":

Start tracking it.

Seriously. Get a pen, pencil, maybe your email box out, and start tracking just the projects you are on or the 'huge ones' swirling around you. This is easy enough to do in excel. If you want to get really fancy, start recording when the project was created and predicted due date, along with when the due-date slips occur, and how much much they slip by.

It turns out that this trivial-to-gather data can be extremely valuable when used to predict the performance of future projects.

More on that next time.