Schedule and Events



March 26-29, 2012, Software Test Professionals Conference, New Orleans
July, 14-15, 2012 - Test Coach Camp, San Jose, California
July, 16-18, 2012 - Conference for the Association for Software Testing (CAST 2012), San Jose, California
August 2012+ - At Liberty; available. Contact me by email: Matt.Heusser@gmail.com

Thursday, September 02, 2010

On Test Estimation - II

Another post I made to the Agile-Testing Group recently:


Here's a simple estimation excercise. My honest advice is don't just read it; actually try it. It takes about two minutes.

To start, think about the space between the bottom of your feet and your knees. Write it down. Then think about the space between your knees and your middle. Write that down.

Then estimate the and write down the size of your torso, then your neck, then your head.

Next, add up all five numbers.

Now compare that to how tall you /actually/ are.

That difference - between how you imagine things and how they actually are - is a picutre difference between task estimates and how long things will actually take.

Except of course, you can see and touch your body and it's been about the same height for decades, whereas code is new and fresh and symbolic and 'tests' are an even-more abstract concept.

When you think about it, tests are a first-order derivative of the code itself. Also, most testing is exploratory in nature, EG early predictions are not the best predictions.

So would I be reluctant to make task estimates on a testing task, given the typical American shorthand that estimate==commitment? Certainly.


I like to think of this as the test estimation rabbit hole. First, we have to have the bad news that test estimation is conceptually impossible.

Then we figure out how to do it anyway.

More to come.

Wednesday, September 01, 2010

On Test Estimation - I

I posted this yesterday to the Agile-Testing List, thought I would share it here as well:

--- In agile-testing@yahoogroups.com, "daswartz@..." wrote:
>
>
> Can you help us understand why the QA people care whether
> you estimate in hours or points? I'm sure they have a reason, which
> should help us better answer the context for your question.
>

I'm not the Original Poster, but consider you are testing feature X. You break it down into tasks and say it will take 5 hours to "test."

The first build is total garbage. You can't even click the submit button.

The next day, you get a new build. You find five bugs. You get a new build
late in the day - four of the five bugs are fixed, and you find three new ones.

You get the fixes in the morning on day three. You find another bug. At noon,
your boss comes up: "You said this would take five hours to test and you are on
DAY THREE of testing? Wassup with that?"

---> Bottom line, there are elements in how long it takes to do testing beyond
the testers control. It's generally possible to estimate a test /cycle/ with
some accuracy, but estimating the entire test /process/(*) is rarely possible
unless you know the devs very well and have had some stability in delivered
software quality for some time.

Estimating in terms of points 'smooths' those gaps.

That's one possible explanation, anyway ...

--heusser
(*) - Yes, this pre-supposes that testing is a separate and distinct activity, I
know, we should be involved up front, whole team, etc. I'm with you. But
you gotta walk before you can run. Let's have that discussion on a separate
thread, ok?

Monday, August 30, 2010

Sheep of a different fold

For a few years now I've been listening and reading to the work Mary Poppendeick has produced with increasing appreciation. Then last year I had the opportunity to interview Mary and Tom for InformIT, as part of InformIT's coverage of the Agile 2009, where Mary was giving a keynote speech.

(Mary and Tom) and I have very different backgrounds. We worked in different kinds of organizations and our careers and interests took us in very different directions.

Yet here was this other person that had both studied the history of the philosophy of management -- and studied the actual effects of those ideas in practice -- and come to the same conclusions as I had.

For that matter, the way that the Poppendeick's approach the subject is different than my stuff, and I think worth studying. So when I found out that they had a google tech talk, I just had to link to it here:



I've spent tens, if not hundreds of thousands of words trying to explain the ideology of "process improvement" and some of my concerns about it. For a quick summary introduction, I gotta say, this video by Mary is shockingly good. Throw in a copy of The Management Myth by Matthew Stewart and you end up with a very good survey of the literature in two source documents.

sigh.

If you need me, I'll be in the corner, licking my wounded pride, trying hard not to cry.

:-)

Seriously, this is a good stuff, and I am pleased to recommend it.

UPDATE: A cursory glance at Poppendeick LLC website finds several 'sound byte' level things that you might take issue with in regards to /testing/. My advice: Ignore the sound bytes that are so easy to misconstrue; watch the video instead. Check out what she actually /says/ about software development, management, and leadership. I expect it will resonate with you. It did with me.

Wednesday, August 25, 2010

What Matt has been up to

Woa. Some cobwebs on Creative Chaos, eh? Seems like Matt Hasn't been very busy, doesn't it?

Gosh, I sure hope you don't think that. Let me bring you up to speed:

a) I've split up my blogging into two places - here (always free, no-registration required) and for the Software Test Performance Collaborative - still free, registration required.

b) That blogging now includes recruiting, running and managing a weekly podcast "This Week in Software Testing" - also free for the new shows. To get the old content, you'll need to be a paid member of SoftwareTestProfessionals.Com. Don't want to pay? You can download every episode as they come out, or you can participate in various contests on the blogs. Or write an article for the magazine.

c) I'm still micro-blogging on twitter under the username mheusser.

d) I'm still producing a column for the magazine, now known as "Software Test & Quality Assurance" magazine, or STQA. After two years of writing a encyclopedia-style column were we defined key terms used in the "theme" of the issue, we decided to shake things up a bit and write an interview column. Each issue we'll gather questions from the community for a 'test expert' and have them answer. For the August issue we interviewed Michael Bolton; the extended interview is now available on-line. I'm at the point now where I need recruits to interview, and shortly after that, I'll need questions ...

e) Beyond working with the fine folks at STP, occasionally I get a bit of time to work with other publishers, including the folks at SearchSoftwareQuality. That includes a few podcast-style interviews, a tutorial on the Selenium IDE, a good place for classic testers to learn about Selenium, as well as a two-part tutorial on Selenium RC, which is a start for programmer-types. (Link to Part I and Part II here). I also just wrote a small piece on Effective Bug Reporting Techniques for SSQ.

f) We finally put the Conference for the Association for Software Testing (CAST 2010) to bed last night with a conference retrospective. We held it in Grand Rapids, Michigan in early August. Instead of presenting, I helped out with the local logistics, recruiting some sponsors, and organizing and funding the evening receptions. CAST 2011 will be in Seattle, Washington. With Jon Bach as the conference chair and James as the program chair, I suspect it will be amazing.

g) With one conference to bed, it's time for me to worry about the next one! STPCon is going to be October 19-21 in Last Vegas, Nevada. It starts the 17th if you grab a two-day pre-conference tutorial. I'm a "track chair" for the hands-on track sessions, I'm running one of the track sessions, running a panel on how to decrease costs in testing, and organizing a lightning-talk like session. Oh, and there will likely be a Monday night reception.

h) Day job! Full time as a member of the technical staff specializing in test for Socialtext. At least I managed to skip the commute, otherwise this stuff would be impossible.

i) It's about time for me to re-start teaching religious education for fourth and fifth grade at my Church during the school year, plus coaching soccer for this fall. (See, I have a life outside of work. Really. Occasionally. Sorta.)

j) I just wrapped up a two-year night teaching position at Calvin College. It was really great, but due to a-i, plus not commuting into Grand Rapids anymore, something had to give. I have small children at home; it would be nice to occasionally see them.

... and then I went crazy.

No, at least semi-seriousy. Based on a discussion on the LinkedIn Discussion list, I just signed up to be the lead editor on a collection of essays on how to reduce he cost of software testing to be published by CRC Press in early 2011.

We've got a good team. We had some solid progress before we signed the contract.

But I did just sign and email the contract last week, and our completed, publisher-ready draft is due Nov 1st.

More to come; at the very least, I'll try to blog pointers to interesting work elsewhere.

But forgive me if I haven't been blogging here much. As I hope you can see, I've been ... kinda busy.

And that was before I went crazy. :-)

Monday, July 26, 2010

Kaner On Testing

Those of you who read this blog know I've probably spent tens, if not hundreds of thousands of words discussing the applicability of hard metrics to the management of software development.

You likely know that I'm not keen on it.

Yet I struggle to make the point sharply and quickly. Cem Kaner wrote something on metrics today that summed it all up in a hundred words or so:

Capers Jones sometimes talks disparagingly about the (claimed) fact that 95% of American software companies have no metrics program. On the surface, this sounds terrible. But what I saw as a consultant was that most software companies have tried a measurement program, or have executives with lots of experience with metrics programs in other companies. The problem is that their experiences were bad. The measurement programs failed. Robert Austin wrote a terrific book, Measuring & Managing Performance in Organizations. When you start measuring something that people do, people will change their behavior to make their scores better. People will change what they do to get better scores on the measurements—that’s what they’re supposed to do. But they don’t necessarily change in ways that improve what you want to improve. Often, the changes make things worse instead of better (a problem commonly called “measurement dysfunction.”) This problem happens more often, and worse, if you use weak, unvalidated metrics. I keep meeting software consultants, especially software process consultants, who say that it’s better to use bad measurements than no measurement at all. I think that’s’ a prescription for disaster, and that it’s no wonder that so many software executives refuse to harm their businesses in this way.

I thought it was brilliant.

If you want more, you can read the source that quote comes from - part I of a series of interviews with Cem on uTest.com.

Or come to the Conference for the Association for Software Testing - CAST 2010 - next week in Grand Rapids, Michigan, where Cem is giving a keynote-level speech.

UPDATE: I've been thinking about it, and it certainly depends how you define your metrics. For example, one kind of measurement that I am in favor of is the "slip chart." In other words, you look at every deadline the team has committed to and how late they are, and you figure the rough percentage of how late they /always/ are. With that, you can predict when you will really be done. The folks in the extreme programming community codify this into story points and burnup charts, and that's fine. I don't have a problem with these used as approximations; as first-order measurements. The problem comes when they are reduced to 100-word silver-bullets without the context of how they can be used well.

So I wouldn't say I'm totally opposed to metrics are part of a balanced breakfast. I'm just leery of the common ideology of measurement by numbers alone, without context.

Tuesday, July 20, 2010

Is Your Software Development Organization Agile?

Elisabeth Hendrickson gave a wonderful keynote on Agile-Testing at STAREast this year, and my Friend Dan Mondello took her definition of Agile and codified it with an article. It's a nice read.

We were sitting together for the keynote, and Dan threw in a picture of us at the bottom of the article. You can see Selena Delesie and Lanette Creamer at the left of the photo, Dan at right, and me in the middle.

If you're not interested in the article, check out the photo. That someone as dopey as me has managed to have some modest amount of success in this field should be encouragement to dopey people everywhere! :-)

Wednesday, July 14, 2010

Selenium IDE

The folks at TechTarget just asked me to write an article on Selenium IDE, the integrated, simple, easy-to-use, free browser-driving automation tool for FireFox.

And they just published it!

You can get the article right now at http://searchsoftwarequality.techtarget.com/tip/0,289483,sid92_gci1516589,00.html. (Free Registration Required)

Tuesday, July 13, 2010

Matt's Big CAST Announcement - Part II

Did you know that CAST will be serving breakfast, lunch, and snacks? They are come free with your conference registration.

More than that, we are currently working on sponsors to provide pop and snacks at dinner every night -- Monday, Tuesday, and Wednesday. And it's possible those turn into dinners.

If you are attending CAST on your own dime, you'll be able to load up on free food and only have to pay for a minimal number of dinners. Likewise, if the company is sending you, between hotel discounts and all the food, the cost will be significantly less than your typical "Big Box" conference with the sixty dollar dinner buffet. You can practically afford to send two for the price of one. I will be personally sponsoring snacks in the Rebel Alliance hospitality suite on Tuesday night. Don't say I never gave ya nothin'. :-)

Of course, since CAST doesn't work with rebels the way, say, a 'STAR' conference might, we may go for a different theme. "The CAST Aways" probably works. Just don't call me "little buddy" and hit me with your hat!

Other stuff - there's a tool called "Is the website down or is it just me?" that seems handy-dandy, and I've got a lot more blog material up at the Software Test Professionals site. If you're hungry for posts from me, check out that site. I'll have a blog post every week (or more), plus, new, a "This Week in Software Testing" podcast up once a week or more.

Friday, July 02, 2010

Matt's Big CAST Announcement - Part I


So CAST - the Conference for the Association for Software Testing - 2010 is coming up, and it's going to be about 40 miles from my place, at the Prince Conference Center at Calvin College.

Last time I checked, the Prince Center was very close to booked solid; people keep asking me where to stay.

I've got two concrete suggestions for you:

#1 Country Inn and Suites - Here's the details from an email forward for 35% off!

Book your stay by July 13, 2010 and enjoy a 35% discount at participating Country Inns & Suites By CarlsonSM hotels when you stay at least two consecutive nights during select dates between July 14 and August 31, 2010.

See all participating Country Inns & Suites By Carlson hotels or check out participating hotels in your favorite regions: Hurry, these great rates are only available for a limited time! Book your stay at countryinns.com today and earn bonus Gold Points® for every online booking.




You'll want the Country Inn and Suites in Grand Rapids, Michigan, on East Beltline Road. It's about 3 miles north of the Prince Conference Center.

#2 Any Marriott You'd like

Specifically, the FairField Inn Grand Rapids, which is maybe 1.5 miles from Prince, probably more like 1. If you really feel like pushing it, you could sign up for a Marriott Rewards credit card, get the points and certificate, and use those. It should be about enough combined for two free nights stay -- but read the fine print. The points often aren't credited to you until the first statement is cut. So you might want to stay at Mariott next time, and use the 35% off the Country Inn and Suites this time.

I hope that helps.

Technical Debt: Refired

Phil Kirkham has the cover story of T.E.S.T. magazine this month, talking about technical debt. It's a good article and I recommend it. One thing I like about it is that Phil tries hard to provide a framework for thinking about tech debt, that he borrows from Martin Fowler.

All of it reminds me of the technical debt workshop we ran at Calvin College in 2008. Not a whole lot poured out of that workshop -- it was only two days long, and we spent most of the time trying to come to get to the 'gelled' state so we could make progress. Ron and Chet weren't keen on the term, Chris McMahon came out against analogy shortly thereafter. There wasn't a lot published afterward; I saw a few proposals go to magazine editors but they weren't wild about the formats we proposed. There were a few good presentations we recorded, but I'm afraid there's lots of editing required, and the videos from the event remain locked on a hard drive on my desk.

It wasn't until a week after the conference, on the e-mail discussion list, that we started to see some of the collaborative, building comments I had hope to have during the conference.

Perhaps, if it had been three days, we'd have gotten there. Perhaps, if it had been three days, I'd be saying "if it had only been four days." I don't know.

Using metaphors to describe our work does have certain risks, but I still think that in many cases the tech debt metaphor can have more value than the risk it creates.

It may be time for me to start writing about this again -- or considering a 2011 or 2012 workshop. I'm not sure.

In the mean time, I've got three other new projects. First, a series of interviews with testers called "Testers at work", currently on the back-burner. Second, a book project on changing the cost/value ration of software testing, that i've announced here, and third, a (near) weekly series of podcast interviews with testers I'm nick-naming "This Week in Software Testing", or TWiST, the first of which is up here.

In other news, you can now catch my blogs in two places -- both here and at www.softwaretestpro.com/blog. The majority of my blogging will likely be at STP, but if the topic is the kind of thing that needs a disclaimer, you'll likely find it here.

More to come.

"Welcome all my friends to the show that never ends; I'm so glad you can attend. Come inside, come inside ..."