Tomorrow Through the Past

(Continued Apologies to Robert A. Heinlein)

This year, at the Conference for the Association for Software Testing (CAST 2011) I was a little sad, but not surprised, to hear so many people say thing like "Gosh, I had never heard of this Context-Driven thing. I've got a lot of reading and catching up to do when I get home."

Actually, that last part -- the lot of reading and catching up to do -- that is kind of encouraging. To be involved in building that next generation of super testers ... that's pretty cool.

It turns out this problem is not unique to software testing.

In the 1980's, when Tom Demarco and Tim Lister wrote PeopleWare, they pointed out that while the typical programmer might have books on programming syntax on their desk, the typical programmer they interviewed had never actually read a book on programming style, method, or methodology.

This month, on the ASQ blog Paul Borawski notes that perhaps thirty percent, or less, of the attendees at a typical American Society for Quality (ASQ) have heard of W. Edwards Deming, the champion of quality. He asks, in essence "Are we forgetting our history?"

Now ASQ is not talking about software test history, but instead the greater history of the Quality Movement -- specifically, quality in manufacturing.

It's kind of a big deal.

Consider, for example, Deming's Seven Deadly diseases of management. I'll quote the first five, using John Hunter's wording:

1. Lack of constancy of purpose


2. Emphasis on short term profits (Overreaction to short term variation is harmful to long term success. With such focus on relatively unimportant short term results focus on constancy of purpose is next to impossible.)


3. Evaluation of performance, merit rating or annual review (see: Performance Without Appraisal: What to do Instead of Performance Appraisals by Peter Scholtes).


4. Mobility of top management (too much turnover causes numerous problems)


5. Running a company on visible figures alone (many important factors are "unknown and unknowable." This is an obvious statement that runs counter to what some incorrectly claim Deming taught - that you can only manage what you measure. Deming did not believe this and if fact saw it as a deadly disease of management)

When I worked at a Fortune 500 company, and even mid-sized companies with as few as 400 employees, every single one of these deadly diseases was considered a "best practice."

These practices were institutionalized. For example, one company had a web-based system to perform your annual reviews on. If you did not fill out your forms by a certain date, including your goals, targets, and management-by-objective), you were guaranteed no raise.

This was company wide.

Bear with me here, I'm seeing a trend.

Something Rotten in Quality-Land

You see, something happened after the quality revolution hit Japan, and Detroit had to struggle to keep up. Top management saw value in the Quality movement, but they were too busy with the busy business of production to do anything about it. So, at best, "quality" was delegated to some fresh, young engineer in the rear rank. "You go for it, Joe." they said "you're our quality guy."

At worst, it was shunted to a Vice President, who shunted it to a director, who created a manager of quality, who continued to delegate it until "quality" became the guy who forgot to step backwards.

Or, perhaps, something entirely different happened; quality was re-interpreted as "compliance", and the company created a "quality system" to comply with an ISO standard.

We could debate about the value of ISO-9001 for manufacturing; it certainly has it's proponents. My point is that Deming laid down seven deadly sins of management, and, twenty-odd years later, the top five, if not all of them are common, institutionalized practices in corporate America.

PLUS we have these people who are considered quality professionals, doing ... something ... working within a system that has it's thinking in diametrical opposition to the principles that Deming was proposing.

I know how that goes.

The Bad News

Being the delegated quality guy when management follows a different ideology and wants "that quality stuffs" but isn't willing to change or pay for it, yeah, that's hard. I've been that guy, and it's not much fun.

It's worse, however, to be titled the "quality guy", working in that same system, but ignorant of the craft. Again, we're stuck with at best you might make some small, occasional, modest improvements, at worst quality continues to get a bad name. The most likely case is that your organization will become infected with questionable metrics, unhelpful process documents, and the occasional template.

yuck.

I see similar issues in my own tiny sub-field of software testing, mostly in the area of test automation.

I have seen organizations implement test automation, expecting test cost to go down ... only to see it actually increase.

This leads me to the maxim "When you automate tests, testing costs go up."

Before I make a wild, unsubstantiated claim like that in public, I wanted to check in with a few people. A discussion with my friend, David Christiansen helped me clarify the statement a bit.

I don't mean to criticize all test automation; I'm very excited about test automation at the developer-level. Even customer-facing tests can be done well -- if you have a tight feedback loop, or have the developers doing the automation, so if they change a GUI field, they also know to change the GUI test.

No, instead I'm talking about a very specific set of circumstances. The term David used was "when the stars align."

When

* Management sees value in test automation, but the developers are too busy to do the work

* So they delegate the development of the automated GUI tests to a specialist, who does the work after development.

* These tests are designed to run at the press of a button, both exercising the GUI and providing analysis and results.

* The feedback loop between development and test is long; perhaps the tests only run as a suite before release. After weeks of growing differences between the systems, the automated tests with report failures that actually 'just' changes. These differences need to be investigated and

* And the company already has a strong, disciplined testing in place, using an approach that is at least partially exploratory

* And Management expects costs to go down

Under these conditions, I see the cost of testing going up, generally without much increase in velocity (features delivered over time) or defect rate. In most cases, velocity goes down.

That is to say, I see companies paying more for software testing and getting worse results.

But ... why?

Now I don't want to paint with too broad a brush here. People have different reasons for the decisions they make; the best I could do, maybe, would be to provide some generalities; rules of thumb that are often wrong.

Perhaps 'suspicions' would be a better term.

Did you notice the pattern between the ASQ and Software Testing? We have people who were successful doing things a specific way, and have built a worldview around that ideology. They hear about an opportunity to improve, and grab at it ... then re-interpret that opportunity around the existing worldview.

Then they outsource the work, but put constraints upon the work.

"We want that 'quality' stuffs, but don't you dare touch my annual review!"

(Come to think of it, we have a very similar problem in the United States Government. But I digress.)

But how do we fix it?

I'm afraid I don't have a "mary poppins" answer to this one. It's a hard problem, and it's far to easy to say something like "the solution is dialogue and education."

By dialogue, I mean conversation.

Companies that adapt these strategy do so because they believe they will work -- and that may be possible. So when these conversations start, we can get trapped in all kinds of different mistakes. You might say "test automation" and the other person thinks developer-testing, which I'd agree can have a lot of value. Or you might say "costs go up" and deeply offend the other person. Or they might feel insulted or condescended to.

Most importantly, these conversations are best done by invitation.

By reading my blog, you asked for my opinion -- even if you disagree. Yet many times our opinion is not ask; instead, someone in authority is handing out directions.

Responding to that direction with integrity can be a challenge for all of us; I'm working on it too.

I have a number of personal initiatives for 2012, but one of them that is just starting to heat up is "Test Coach Camp", which I hope will produce more concrete guidance on Crucial Test Conversations.

Our industries are young, the pull of popular culture and cliches is strong.

We've got work to do.

Let's get to it.

My Independence Day (Or Something Like It)

Friends of the blog have probably noticed that, around May of this year, I went independent.

There wasn't really a big announcement. In April I was hit by a restructure and reduction at Socialtext. Given the market and venture capital environment, the CEO did what he had to do -- I left on good terms.

The timing was pretty good; I immediately called up my editor friends and announced that I had sudden availability, plus I finally had time to get more involved in uTest, catch up on some reading, and, just maybe, see my family a little bit more.

Yes, I also started a job hunt.

After about the second week, I started to realize that I wasn't really looking for a full-time, employee position. Oh, yes, I'd probably take a wonderful position if it came around, but it seemed that every company I was talking to wanted me to move to Florida, or Seattle, or California, and they wanted me to move right now and bet my families future on the hope that the job, or at least the job market in that area, would stick around. There were a few remote positions, but they generally had timing demands and full-time salaried expectations; no one wanted to try a contract relationship first, to see if we might be a good fit.

By the end of the second week, when I called the unemployment hotline, and it asked if I was looking for work as a full-time employee, I realized the answer was 'no', or at least 'not only.'

It was time to make Excelon Development a full-time concern, or at least, it was time for a new kind of experiment.

Four months later, I seem to be makin' it.

At the same time, I've learned a whole new world. I've learned about liability insurance, the self-employment tax, anticipated quarterly tax payments, and more.

One of the more interesting ones, at least in the United States, is the Alternative Minimum Tax, or AMT.

Essentially, the AMT tax is the minimum tax you can pay off your gross income, no matter what you deduct.

If you are ever considering going into a line of work where you have decent compensation, but also decent expenses -- say travel expenses when consulting -- it's certainly something to think about.

Trying to take lemons and make lemonade, I took that research and published an article on the alternative minimum tax for Adaptu.com.

Please check it out.

If you tell me what'd like you to read in the area of personal money or small business management, it's all grist for the mill of future articles. (I've been thinking of starting a money management blog, which would give me control, but I don't think I have the energy to sustain it.)

Next on the plate is examining the true cost, and true value, in going back to school at night.

More to come.

CAST 2011 Thoughts

I'm in Colorado on vacation, but waiting in a auto shop to get our minivan repaired.

Why are we in Colorado? My family is taking the long way home from the Conference for the Association for Software Testing -- yes -- driving from Seattle, Washington to West Michigan over a couple of weeks. It seems the rocky mountains were a bit much for our transmission ... at least the second time through.

I've been meaning to do a wrap-up post on CAST, and it seemed like a good time.

First off, CAST was amazing. Jon and James Bach did an impressive job attracting some of the best minds around, then created a format to set them free. The highlight for the conference for me was not the few talks I gave, but instead the Test Competition on Monday night. From 6PM to 10PM we self-organized into teams, attacked a piece of software, filed bugs, and, at the end of the night, produced a test report.

As an organizer of the conference (I helped organize the Emerging Topics Track, along with Pete Walen), I was ineligible to win a prize. If I joined a team and competed, that would make my team ineligible for any cash prizes -- and the prizes totalled fourteen hundred dollars. So when the Miagi-Do students formed a team, I tried to stay hands off. I thought I'd hang out with some folks, maybe grab a beverage with Pete Walen.

I'm afraid it did not turn out that way.

Instead, the members of the team -- Ajay Balamurugadas, Michael Larsen, Markus Gaertner, Adam Yuret, Justin Hunter, and Elena Houser -- insisted that they'd rather test with me than win a cash prize.

How do you turn that down?

So I played on the team. Along the way, several people expressed concern; after all, here I was, a conference organizer. Didn't I know the rules? Didn't my team know the rules?

It turns out, we did. We were playing for something else ... for love of the game.

I felt I could not refuse. Our team did well. Markus, Michael, and Elena all have blog posts on it; Adam tells me he has blog post in the works.

The rest of the conference, though, was pretty much a blur.

Oh, in a sense, I accomplished a big personal goal. I was elected to sit on the board of directors for the Association for Software Testing and gave my three talks without having some sort of terrible accident; most people seem to think they were helpful.

In another sense, I'm afraid my CAST experience was a mix of indebtedness and regret.

I feel in debt because of all the work people did on my behalf to make things come out well. While I proposed Emerging Topics and put the framework in place, it was Pete Walen that did the heavy lifting and acted as facilitator. Then there was the audio for my talk, that Ben Yaroch and Tim Coulter put together on no-notice. (Ben found me an audio cord to go from my computer to the project, that I failed to give back. Then I found my presentation was in a different room with different equipment, and Tim, my facilitator, ran and bought me speakers during my talk, before I needed audio. Did I mention that I was on a mac and Jon Bach brought the converter to get the video connection to work?)

Seriously. Amazing.

Then there was Michael Bolton, Jon Bach, Markus, Justin Hunter, Anne-Marie Charrett, Selena Delesie, and a few people from the audience that participated in the "How to Reduce the Cost of Software Testing" panel on Tuesday:

If you are noticing the same names again and again, you're on to something. Like I said, in debt. The panel self-organized, the whole thing came off without a hitch, because of the contributions of the panel members and the interaction with the audience.

Regret because of the people I missed. Despite arriving on Sunday at 4:30PM and sticking around until Tuesday at 5:00PM, there just wasn't enough time to catch up with everyone.

I did manage to have a few minutes with Gary Masnica, only because he picked me up at the airport. I got to talk to Griffin Jones a bit, and most of the people above because, well, we had conference business to run. Yet between Selena, Lanette, Lynn Mckee, Anne-Marie, and Pete, I have to admit my conversations were the sort of quick, dash-and-run conference conversations that I am really not a fan of.

I barely got to shake Felipe's hand, I couldn't make Greg McNelly's session and barely got time to poke my head into Markus's. I even missed Gerodie Keitt's testing improv session and didn't really get a chance to talk to him, either.

In other words, if you were disappointed at my participating in CAST, well ... so was I.

I blame Jon and James: They assembled a group of people so awesome that it was not humanly possible for me to catch up with them all. I didn't even mention the keynotes yet, or other conference material ...

I suppose, if the bar for conferences continues to be set this high, that's something I'll just have to get used to.

Still, the good news is we'll get more chances. Many of these folks will be at STPCon, where I'll be staying four days.

Plus there is always 2012, more conferences, and, just maybe, Test Coach Camp.

More to come.

I can't wait!

Quality in the Executive Suite

For this months "View from the Q", the people at the American Society for Quality recorded an after-talk chat with Dr. J.J. Irani of Tata Group, India's single largest company.

Of course, there's the typical stuff I disagree with in the talk, your typical appeals to standards like the Malcolm Baldridge award, and this idea that it's the government's business to reward companies that comply to some sort of standard for performance improvement.

Yet in this case, I think the things that we agree on are more interesting -- or at least worth talking about.

Two points stood out for me, in the second and third video.

In the second video, J.J. claims that in order to have trust, organizations must have aligned values. He goes on to say that Tata will walk away from a bad deal.

In the third, he claims that companies have a sort of obligation to contribute to the communities in which they operate. To use his analogy, you can't have a tree sprouting up in a desert -- so Tata needs to go give back some financial capital periodically. According to J.J., every time they have done this, it has paid dividends. (For example, helping to fund a local school, thus making the pool of qualified applicants larger, making it easier to hire and, since you have the people faster, easier to get things done. That's my words, not his.)

I happen to agree with both of these. Back home in West Michigan, we have a company named Bissel that makes vaccum cleaners. A few years ago, they decided that, in order to survive, they needed to sell the plants, lay off the line workers, and move manufacturing to Mexico. It is ... sad.

It's easy to be critical of Bissel, but let's face it -- Bissel lasted ten years longer than General Motors did in Flint, and twenty years longer than most of the American Textile Industry.

I can understand having to make tough decisions like that, and I don't mean to minimize it.

They tried.

But I keep thinking about that tree trying to grow in the desert.

Something's happening in the American Economy. Something fundamental.

Have you noticed what the economists are saying about a recovery? That it will be twelve to eighteen months, that we'll have growth, but that it will be slow?

Have you noticed that those are the same things they were saying in 2010?

And 2009?

And 2008, when we had the last "crisis?"

It's pretty simple math. Companies keep growing through mergers and acquisitions.

Every time you have a merger, the acquiring company now has two IT department, two HR departments, two legal departments, two data centers, two finance departments ...

When the company merges the departments, it will find duplication. Remove the duplication, have the sales folks share customer lists, cross-sell products, and, boom, mo' money.

The only problem is all the laid off people.

This isn't a cyclical problem. It is a systemic problem.

Getting executives to sit down and talk about trust and long-term consequences might not solve it, but hey, it's a start.