Kaner On Testing

Those of you who read this blog know I've probably spent tens, if not hundreds of thousands of words discussing the applicability of hard metrics to the management of software development.

You likely know that I'm not keen on it.

Yet I struggle to make the point sharply and quickly. Cem Kaner wrote something on metrics today that summed it all up in a hundred words or so:

Capers Jones sometimes talks disparagingly about the (claimed) fact that 95% of American software companies have no metrics program. On the surface, this sounds terrible. But what I saw as a consultant was that most software companies have tried a measurement program, or have executives with lots of experience with metrics programs in other companies. The problem is that their experiences were bad. The measurement programs failed. Robert Austin wrote a terrific book, Measuring & Managing Performance in Organizations. When you start measuring something that people do, people will change their behavior to make their scores better. People will change what they do to get better scores on the measurements—that’s what they’re supposed to do. But they don’t necessarily change in ways that improve what you want to improve. Often, the changes make things worse instead of better (a problem commonly called “measurement dysfunction.”) This problem happens more often, and worse, if you use weak, unvalidated metrics. I keep meeting software consultants, especially software process consultants, who say that it’s better to use bad measurements than no measurement at all. I think that’s’ a prescription for disaster, and that it’s no wonder that so many software executives refuse to harm their businesses in this way.

I thought it was brilliant.

If you want more, you can read the source that quote comes from - part I of a series of interviews with Cem on uTest.com.

Or come to the Conference for the Association for Software Testing - CAST 2010 - next week in Grand Rapids, Michigan, where Cem is giving a keynote-level speech.

UPDATE: I've been thinking about it, and it certainly depends how you define your metrics. For example, one kind of measurement that I am in favor of is the "slip chart." In other words, you look at every deadline the team has committed to and how late they are, and you figure the rough percentage of how late they /always/ are. With that, you can predict when you will really be done. The folks in the extreme programming community codify this into story points and burnup charts, and that's fine. I don't have a problem with these used as approximations; as first-order measurements. The problem comes when they are reduced to 100-word silver-bullets without the context of how they can be used well.

So I wouldn't say I'm totally opposed to metrics are part of a balanced breakfast. I'm just leery of the common ideology of measurement by numbers alone, without context.

Is Your Software Development Organization Agile?

Elisabeth Hendrickson gave a wonderful keynote on Agile-Testing at STAREast this year, and my Friend Dan Mondello took her definition of Agile and codified it with an article. It's a nice read.

We were sitting together for the keynote, and Dan threw in a picture of us at the bottom of the article. You can see Selena Delesie and Lanette Creamer at the left of the photo, Dan at right, and me in the middle.

If you're not interested in the article, check out the photo. That someone as dopey as me has managed to have some modest amount of success in this field should be encouragement to dopey people everywhere! :-)

Selenium IDE

The folks at TechTarget just asked me to write an article on Selenium IDE, the integrated, simple, easy-to-use, free browser-driving automation tool for FireFox.

And they just published it!

You can get the article right now at http://searchsoftwarequality.techtarget.com/tip/0,289483,sid92_gci1516589,00.html. (Free Registration Required)

Matt's Big CAST Announcement - Part II

Did you know that CAST will be serving breakfast, lunch, and snacks? They are come free with your conference registration.

More than that, we are currently working on sponsors to provide pop and snacks at dinner every night -- Monday, Tuesday, and Wednesday. And it's possible those turn into dinners.

If you are attending CAST on your own dime, you'll be able to load up on free food and only have to pay for a minimal number of dinners. Likewise, if the company is sending you, between hotel discounts and all the food, the cost will be significantly less than your typical "Big Box" conference with the sixty dollar dinner buffet. You can practically afford to send two for the price of one. I will be personally sponsoring snacks in the Rebel Alliance hospitality suite on Tuesday night. Don't say I never gave ya nothin'. :-)

Of course, since CAST doesn't work with rebels the way, say, a 'STAR' conference might, we may go for a different theme. "The CAST Aways" probably works. Just don't call me "little buddy" and hit me with your hat!

Other stuff - there's a tool called "Is the website down or is it just me?" that seems handy-dandy, and I've got a lot more blog material up at the Software Test Professionals site. If you're hungry for posts from me, check out that site. I'll have a blog post every week (or more), plus, new, a "This Week in Software Testing" podcast up once a week or more.

Matt's Big CAST Announcement - Part I

So CAST - the Conference for the Association for Software Testing - 2010 is coming up, and it's going to be about 40 miles from my place, at the Prince Conference Center at Calvin College.

Last time I checked, the Prince Center was very close to booked solid; people keep asking me where to stay.

I've got two concrete suggestions for you:

#1 Country Inn and Suites - Here's the details from an email forward for 35% off!

Book your stay by July 13, 2010 and enjoy a 35% discount at participating Country Inns & Suites By CarlsonSM hotels when you stay at least two consecutive nights during select dates between July 14 and August 31, 2010. See all participating Country Inns & Suites By Carlson hotels or check out participating hotels in your favorite regions: U.S. - Midwest U.S. - Northeast U.S. - Southeast U.S. - Central Midwest U.S. - Southwest U.S. - West U.S. - Mountain Costa Rica Hurry, these great rates are only available for a limited time! Book your stay at countryinns.com today and earn bonus Gold Points® for every online booking.

You'll want the Country Inn and Suites in Grand Rapids, Michigan, on East Beltline Road. It's about 3 miles north of the Prince Conference Center.

#2 Any Marriott You'd like

Specifically, the FairField Inn Grand Rapids, which is maybe 1.5 miles from Prince, probably more like 1. If you really feel like pushing it, you could sign up for a Marriott Rewards credit card, get the points and certificate, and use those. It should be about enough combined for two free nights stay -- but read the fine print. The points often aren't credited to you until the first statement is cut. So you might want to stay at Mariott next time, and use the 35% off the Country Inn and Suites this time.

I hope that helps.

Technical Debt: Refired

Phil Kirkham has the cover story of T.E.S.T. magazine this month, talking about technical debt. It's a good article and I recommend it. One thing I like about it is that Phil tries hard to provide a framework for thinking about tech debt, that he borrows from Martin Fowler.

All of it reminds me of the technical debt workshop we ran at Calvin College in 2008. Not a whole lot poured out of that workshop -- it was only two days long, and we spent most of the time trying to come to get to the 'gelled' state so we could make progress. Ron and Chet weren't keen on the term, Chris McMahon came out against analogy shortly thereafter. There wasn't a lot published afterward; I saw a few proposals go to magazine editors but they weren't wild about the formats we proposed. There were a few good presentations we recorded, but I'm afraid there's lots of editing required, and the videos from the event remain locked on a hard drive on my desk.

It wasn't until a week after the conference, on the e-mail discussion list, that we started to see some of the collaborative, building comments I had hope to have during the conference.

Perhaps, if it had been three days, we'd have gotten there. Perhaps, if it had been three days, I'd be saying "if it had only been four days." I don't know.

Using metaphors to describe our work does have certain risks, but I still think that in many cases the tech debt metaphor can have more value than the risk it creates.

It may be time for me to start writing about this again -- or considering a 2011 or 2012 workshop. I'm not sure.

In the mean time, I've got three other new projects. First, a series of interviews with testers called "Testers at work", currently on the back-burner. Second, a book project on changing the cost/value ration of software testing, that i've announced here, and third, a (near) weekly series of podcast interviews with testers I'm nick-naming "This Week in Software Testing", or TWiST, the first of which is up here.

In other news, you can now catch my blogs in two places -- both here and at www.softwaretestpro.com/blog. The majority of my blogging will likely be at STP, but if the topic is the kind of thing that needs a disclaimer, you'll likely find it here.

More to come.

"Welcome all my friends to the show that never ends; I'm so glad you can attend. Come inside, come inside ..."