Schedule and Events

March 26-29, 2012, Software Test Professionals Conference, New Orleans
July, 14-15, 2012 - Test Coach Camp, San Jose, California
July, 16-18, 2012 - Conference for the Association for Software Testing (CAST 2012), San Jose, California
August 2012+ - At Liberty; available. Contact me by email:

Thursday, June 28, 2007

At my limit ...

About two months ago, I stopped exercising. "Just until I can get over this stressful period." I also gave up my dietary restrictions.

About a month ago, I stopped blogging.

I've been working a lot of hours for one particular company on one particular project. It's modifications to an incentive financial system that produces $15 million of checks per month. I am responsible for gathering requirements, doing design, coding, unit and system testing. I do have someone to bounce ideas off for design/code review, and customer acceptance tests done by the customers.

Last week, two people pointed out that I was getting "snippy" lately.

My desk, house, and, increasingly, yard - is an awful mess.

Tuesday, I noticed that I was biting my nails. Looking at them, I've bit them down far further than is reasonable. "To the bone" is an exaggeration, but not by much.

We shipped the first incremental release yesterday, and, when the interface ran, no regression errors and 3 of 4 major issues were fixed. Whoopee.

Still, I struggle. One on hand, I can't stand the kind of refuse-to-take-responsibility, refuse-to-make-commitments, just-dont-care attitude that I see in some technologists.

On the other, this is killing me.

I need to find a third way.

What does this have to do with Creative Chaos?

Well, In a previous post I mentioned informal measurement systems - like the ones you use to tell that you need a hair cut. Interestingly enough "Is Matt Stressed" is a very simple measurement system - LOOK AT HIM.

Yet, to a formal measurement system, everything is GREAT. Why, we should do more of this!

I didn't start out with a moral, but here it is anyway: If you have a problem where formal measurement doesn't fit, don't force it - instead, use informal measurement. Otherwise, someone is probably going to get burned ...

Thursday, June 21, 2007

Metrics Madness

A Cautionary Tale

Years ago I worked for an established fortune 500 company. At the beginning of each year, executives set goals by which they would be evaluated. These objectives were numerical and SMART – Specific, Measurable, Achievable, Relevant, and Time-Boxed. In order to make sure that the person didn’t do long-term damage to meet a specific goal, the company required at least two goals, or a “balanced score card.” They were also somewhat enlightened in that, within ethical limits, how you met the goal mattered much less than weather or not the goal was met. The company was split into independent sub-units, each with it’s own profit and loss statement – a wonderful source of hard metrics.

If you think about it, the way that company was run, metrics and all, is very similar to one particular point of view for IS Management. The argument goes that if we could only take those ideas and adapt them to software engineering, all would be salt and light.

To answer that argument, I would like to tell you a story.

In the late 1990’s, one of those independent units had a Vice President of Sales, who I will call Joe. Well-deserving of the job, the man was seriously brilliant. Taking over from the last sales VP, Joe reorganized the way sales was done, focusing on selling things which cost less to produce and sold for more money. He also expanded the client base, selling into markets that could see more value in the product or with larger purse strings – thus making sales easier.

By the tenth month of the fiscal year, Joe’s sales team had booked more profit than the “exceeds expectations” goal set at the beginning of the year. Why, with the incentives complete and no further profit possible, the team would be just as well off to take two months off work and start again in January, right?

Well, of course not. The company could always use more money, and besides, Joe was measured with a balanced score card. Because he sold more profitable items, he had met his profit target but not his gross sales target. Unless he could hit the gross sales goal, there would be no bonus and no big raise. Yet, late in the year, most customers had already spent the available budget; what remained was very little.
Dilemma. What’s an ambitious business genius to do?

You probably guessed it – Joe had his team go back to the old products and sell them at a loss in order to hit the gross sales goal. This is dysfunction; having all the metrics right but missing the point. As the anonymous philosopher once said “Be careful what you measure, because you are going to get it.”

In this story we have an established, mature organization trying to metrics right and do the right metrics. They used established accounting and business administration principles, which, compared to software engineering, seem wise and established. Why, Mark Twain once remarked that there are three good ways to mislead: Lies, Damn Lies, and Statistics. If metrics dysfunction can find it’s way into a mature field like business administration, we must realize it is a very real risk for software engineering.

Playing pick a number at the beginning of the year and “managing to the numbers” may be easy, but that doesn’t make it right. Numbers can provide information or evidence to help lead to a conclusion, but without the context, we’re likely to make a mistake. We might either abuse the metrics - like the example above, or misinterpret them - such as the new defects trend line that seems to shrink right around spring break.

Twenty years ago, Tom DeMarco wrote that “You can’t control what you can not measure.” Jorge Amanda points out on his blog1 that without measurements it would be very hard to control weight loss, blood pressure, or cholesterol. Yet he also begs this question:

When was the last time you measured the length of your hair? So how can you control the length of your hair?

Now, take that observation and general systems thinking and apply it to software, and I’ve got two wonderful words that describe it:

Management and Leadership.


Friday, June 08, 2007

The Secret and the Sabbath

The Secret
In the last conference post I promised to explain my system to generating ideas to present or write about.

It's really simple.

First, you know enough of the software success literature to know the way things are "supposed to be".

Then, you pay enough attention to the real world to see how they actually are.

Then, you exploit the difference.

There are a few ways to do this:

"Best Practice A proposed to solve problem number 1. But if you think about it, doing A introduces problems 2, 3, 4, and 5. Now I ask - what problems would you rather have?"

"Fred Brooks says that A is true. Steve McConnell points out B. Kent Beck says C. If A, B, and C are true, then D is also true. And once we know that D is true ..."

"Did you ever notice that BLAH actually happens a lot on software projects, but nobody wants to talk about it? Here's one way I have dealt with that."

---> A great place to start this is to read a lot of Jerry Weinberg's writings, then compare them to, uh, *ahem* the, uh, traditional software success literature.

Anyway, that's the 30 second version. I may build on that at a later date.

The Sabbath
Creative Chaos is a wonderful, fun experiment. It also steals energy and attention from my life - which is fine - if it is a priority.

So here's what I'm saying: I need a rest from blogging to focus on other things. So I'm going to take at least a month, possibly more, and give those things the focus they deserve. (Oh, I may post new speaking engagements, but if you really want to here every new annoucement, join my discussion list - SW-IMPROVE. Volume is very low - about a post a week right now.)

It's not that I don't enjoy blogging. It is that I enjoy it too much. For a little while, I'm going on a forced break. For now, the web can wait.

Most likely, instead of crazy-frequent posts, I'll be back in a month with a post every week (or two?) - and it will be a solid one. But who knows; sometimes you come back from rest, well, rested, and ready to start again.

I hope to see you then!

Dev to Tester Communications

Last week I posted the comic where the tester says "Hey dev guy, your tests suck! They aren't actually testing anything!"

And the dev guy replies something like "You're just being a jerk."

Jon Kohl tells me that I'm only showing one side of the coin; the other one is where the dev guy points out that the tester's automation code sucks.

Ok, so it goes both ways.

So, in that spirit, I've found this comic. Enjoy.

"The SDLC"

There's been some great discussion on software-testing yahoo group lately.

Here's the latest post, by Jeff Fry, quoting myself and Ben Simo:

Ben Simo wrote:
I view the all-too-common resume claims like "full understanding of the SDLC" to be a sign of a lack of experience.

Matt Heusser Replies:
(huge, error-prone heuristic warning here) I find that people who talk in terms like "The Software Development Life Cycle" (SDLC) are talking in models and abstractions they use to make things seem easier.
When someone is all "SDLC this, SDLC that", my warning flags go up.

I much prefer people who talk about what actually happens on real software projects with concrete examples.

Much like Ben says, but I'd go futher: There is a difference between the abstraction and reality. A lot of my work is exposing that difference and helping people deal with it. If someone doesn't seem to grok that difference ...

And Jeff Finally Closes:
This reminds me of one of my favorite Dietrich Dorner quotes (Logic of
Failure, p 55):

By labeling a bundle of problems with a single conceptual label, we make dealing with that problem easier - provided we're not interested in solving it. Phrases like "urgently needed measures for combating unemployment" roll easily off the tongue if we don't have to /do /anything about unemployment. A simple label can't make the complex nature of a problem go away, but it can so obscure complexity that we lose sight of it. And that, of course, we find a great relief.

Tuesday, June 05, 2007

Conferences - V

I promised to detail my strategy to start attending conferences.

Actually, it's pretty simple.

1) Get involved in a local user's group. Meet colleagues, discuss ideas, help other people solve their problems, give presentations and/or join the leadership team. If you don't have a local user's group, found one. (No, it is not that hard - I founded the Grand Rapid Perl Mongers in 1998, a user's group that is still around. How to do that is a different post.)

The value in the user's group is networking, but I disagree with a lot of people about networking. Some think it is about collecting a large rolodex or making acquaintances to 'get' a job or contract. Well, that might work; there are specific techniques people use to jump from lead to lead.

Personally, I don't buy it. Why would anyone want to connect to a leach? To network effectively, you need to get to know other people at least well enough to know what they are interested in. Once you do that, you listen to the problems they have and see if you can help. Eventually, people in your network will look forward to calls from you because they know you will be offering to help them in some way - or at least have something interesting to discuss. Perhaps, some day, a sense of reciprocity kicks in, and they mention an opportunity for you. Perhaps not. The point is, now a lot of people know you and your reputation grows.

---> And you'll use that sidebar reputation in step two.

2) Put your local reputation to good use:

You could ...

2a) Look for a regional conference or a national conference in your area.

If you live in a big city like Chicago, Indianapolis, Denver, or Portland you are sure to find a regional conference staffed by volunteers. This conference is probably close enough that there will be no travel costs and no hotel costs. It will be short enough that you can miss a day or two - a max of three days - not a week. And you can get in free in two different
ways; you can either speak or volunteer to help run the conference.

For that matter, there are a lot of big national conferences that are in the same place every year, like the Twin Cities, Minnesota area, the Orlando Florida Area, the Boston Area, SanFranciso , Orange County California, and so on. Again, you can get in by either speaking or, in many cases, even the big conferences allow a limited number of volunteer slots. Also, the big conferences may be as long as a week, but you can usually sign up for "Just the conference days" or "Just the tutorial days" and only miss three or four days of work.

If you live in the United States or Canada and can't find a local conference, drop a comment on my blog and I will see what I can do.

2b) Eventually, someone in your user's group will know someone else who is hiring. And if you are an active member of the group, they will have some ability to evaluate you, and possibly recommend you. So here's the secret: Go to a company that regularly sends it's technology employees to training. Really, some do exist, especially companies with a rule like "One week
of training every two years."

In my experience, companies that have some training offered to employees are either better places to work or at least tend to pay better.

Plus, by knowing the people that work for the company, you'll know a lot more going on. You'll know the technologies they use, the problems they have, and you'll know if the employees there like it or hate it. In short, you will know if you want the job or not.

2c) Found your own regional conference. Again, I've done this, it's not as hard as it sounds, but this one actually does take some work. If you check out my comments from around November of 2006, you'll see some the results of our after action review from our first year of GLSEC.

(Option 2D is to attend a free peer conference, such as WOPR, LAWST, or IWST, but I suspect again that is an entirely different post.)

3) By the end of step two, you are attending a conference, probably every year. The next step is to move up from a regional conference to a national one. A few ways to do that ...

3a) When you move to the new job, make training a part of the hiring negotiation - at least every-other year. Ask about the training budget. Be specific. Be firm. Don't jump to a company that has recently made commitments and then suddenly had a hiring freeze.

3b) Find a national conference that you can drive to and get accepted as a speaker (or volunteer if possible). Then your company only has to kick in the fee for the hotel room.

3c) Based on your ever-expanding network, find a third job and ... you get the picture.

3d) Start writing or blogging enough to boot-strap demand. Then ask the conference company to cover your travel expenses. (If you speak for two or more hours, they are much more likely to do this. Speak for a half day and you'll get some kind of support, anyway.)

To sum up:
If you make attending conferences a goal in the same way that some people make "increasing salary" a goal, you're pretty sure to get it. If you really want salary, attending conferences is a decent way to get it - you can build connections, reputation, and companies that send people to conferences tend to have more money to spend.

There are possible exceptions.
You could have a niche role small enough that you can't find local conferences. (data modeling or business intelligence, for example) You could live in a "one company town" in the deep Midwest or mountain area. Outside of the US, you could live in a large town with very little in the way of IT jobs, or you might have to bootstrap a development or testing community.

It's late. I'm starting to ramble. Anyway, that's the strategy.

Next time: How to generate ideas ...

Monday, June 04, 2007

Tutorials and Training

In a response to my blog post on training, Mark wrote:

I would worry more about the thrashing effect of a developer testing course which did not focus on a specific developer tool set. Choosing a specific tool set allows the teacher to remove many variables from the discussion so they can rapidly teach concepts like "red, green, refactor" without spending time deciding if pyunit, nunit, junit, TestNG, or utplsql is the right choice for the user.

I agree, and here's what I'm thinking -

A one-day course on "What works in software testing: Or How To be Lazy Without Really Trying" (With Credit Mike Schwern, of course)

Before the course, attendees email a list of the core technologies they work on. Then, we customize the afternoon, perhaps splitting into small groups.

Essentially, the course is "Homebrew Test Automation"++ - as a workshop.

Now, the whole idea isn't fully baked, but developing ideas with people on the cutting edge is part of what I use this blog for. The risks in the format are medium-sized; people REALLY like stable, repeatable training and start to weird out when I customize training on the fly. Such custom training is also hard to get right and easy to get wrong.

Still, that's what I'm thinking about right now. What do you think?

Lightning Talks Wrap-Up

Just a quick after-action review from STAREast --

Overall it was a great experience. Alan Page has his lightning talk up on-line in English Prose; Michael Bolton has his as a powerpoint (look for his May 27 Post), and I integrated mine into the post Conferences II.

One thing I tried this year that was different was outlawing power point; each speaker had to present using only words, markers and paper.

Of course, the audience would expect powerpoint, so for each speaker we had an intro slide on a black background with white letters - name, title, company, and logo. Then after that I had a completely black slide.

Switching to completely black messed with the projector; it thought there was no content. So I had to change the slides to very-very-very dark grey.

Overall, I think it was distracting. Next time, I will either allow powerpoint from the presenters, or have none at all. It was a nice experiment, but the middle road is often not the best one. :-)

Another thing I tried was to shorten then intro/conclusion, run the talks as fast as possible, and, time permitting, shim in a tenth talk. This gave the audience the maximum amount of content for the time.

Sadly, this means we missed a few things (like "Can we get a round of applause for all the speakers?") and I think the audience missed some closure.

So, as great as Michael Bolton's second speech was, I think that in the future I will stick with nine.

Oh, and I give away a noisemaker, which never seems to go well. I need to just go buy a gong.

(Some people who were at STAREast will reply "Matt, what are you talking about? The LT's were great!" And perhaps you are right. Still, I've heard a title for people who aren't self-critical and constantly trying to improve - and that title is 'Amateur' ...)

Friday, June 01, 2007

A thought ...

Meta-Warning: This post is not meant as criticism of any individual or group

I just joined the testdrivendevelopment yahoo group, and saw an announcement for this public workshop by Ron Jeffries and Chet Hendrickson.

Offhand, it sounds really great. Listen to this intro:

Is "Agile" tasting a bit watered down to you these days? Would you like a stiff drink from the bottle under the bar? Then we're the guys to set you up.

Let's go and hang out at the Tech Center Marriott in Denver. You'll join a roomful of people with experience and interest in Agile, where we'll run through a series of exercises that will show you what we mean when we talk about Agile and XP and what software development should really be like. We'll work together so that we can all become more successful and can feel when we are stepping off the path that leads to success.

Super-Cool. Really, this sounds great. I'm excited, and thinking about sending out notes to all around to encourage this, come home and talk about it, and so on.

Then I read on ...

We'd like you to bring a laptop, with Eclipse 3.2, Java 5, JUnit 4, FitNesse, and Subclipse, ready to go. We'll be working in pairs, so if you can't bring a laptop, let us know.

Ah, there's the rub.

There are a bunch of nifty tools in Java (and, to some extent, smalltalk) that all hook together to make a certain brand of agile development work. This includes the magical automated acceptance tests with fixtures.

You come to the workshop, have fun, learn stuff ... then go home to code in PL/SQL or Delphi with none of the tool infrastructure, and you are back to square one.

The over-reliance of tools concerns me a bit. For example, I know of one training company that trains on lean-agile testing that teaches Fitnesse exclusively. I am _CERTAIN_ that a large number of attendees - perhaps a half, or two-thirds, learn about how great things "could be", then go home with no real change in the way they do work.

That's not helping - it's drive-by training. At least Ron and Chet can help ... if you code in Java, and you don't have any significant testing challenges.

This gets me thinking of a workshop day that meets you where you are and helps you build a custom testing framework for your office - to put things in place to do better testing and test automation. A few years ago, Brett Pettichord did a class on "Homebrew Test Automation", but otherwise, I see a gap for this kind of training in the software testing space.

That's not meant as a criticism. It's an opportunity ...