Feedback again

Due to issues with blogger I had to repost this. Notice several newer articles appear below this post.

(Begin old post)
I recently placed an article in software testing in a magazine; it will appear in the February Issue.

I think it needs work, and I wonder if you agree. Would you like a sneak peak in trade for some feedback?


A Box Of A Different Color
A short while ago I had to prepare a testing course for software developers. The development team was responsible for both unit testing and system testing, and programmed in a relatively unpopular programming language based loosely on Pascal. Searching for course materials, the information I found kept pointing back to Java and C++, but the development staff simply did not know or use those languages. In fact, the best material I could find was from a course by Dr. Cem Kaner known as Black Box Software Testing, or BBST.(*)

Next I posted on a software discussion list, actively asking people to recommend open-source materials and what they thought of using BBST materials for software developers. The answers I got back were consistently the same: "No, No, No, you can’t teach a course in Black-Box Testing, go get some material on glass-box testing", and they would recommend a course that ended up being in Java or C++. Occasionally they would recommend a book or class and say something like, "I have heard good things about this but haven’t read it" - it would wind up to be in Java or C++. The existing material would simply not be helpful to the development staff, and I kept coming back to BBST

My next question was to ask why the BBST course was inappropriate. The typical response was "It’s black-box, genius. That’s not the right kind of testing." Others would say "Well, black-box testing is really for people who can’t see the code - it’s not really for the developers."

Both of those answers were unsatisfying to me. They were the definitions of the term, not reasons to reject the course material. One or two people mentioned that black-box testing doesn’t include coverage metrics, which is true, but it turned out that the customer had no requirements for coverage to be measured, and the Dr. Kaner’s BBST course actually includes just enough information about coverage and coverage metrics to meet my needs.

So I looked at the basic skills that the Black-Box course covered: critical thinking, general systems thinking, patterns of software failure, issue isolation, and investigation skills. These are skills that any tester might want to grow, and I found a huge overlap in these skills, regardless of box color. (See Venn diagram, above right)

There were other areas of the BBST that I might want to skip. Bug Advocacy and How to Write a good bug report become less important if the person who logs the bug is the one who is going to fix it - and bugs found in unit testing might not get logged at all. Still, it seemed foolish to through out the entire course when I could just throw out a couple of chapters.

This got me to think about what it takes to be a good black box tester and what it takes to be a good clear box tester.

The developers who would take my course already had the skills on the left. Because equivalence classes work just as well on functions as they do on GUIs, the developers could stand to benefit from any black-box material that would cover the skills in the middle.

If I can screen things from the far right out of the training materials and the developers already have the skills at left, then the differences between Black-Box and Clear Box Testing (or the value of distinguishing the difference) greatly decreases.

At that point, if the difference between the two is not valuable, then why are we splitting them up? On this project, splitting hairs over the color of the box was actually harmful. It caused me, and many others, to shut out potentially valuable learning opportunities that were available, and even free.

Yet the valuable things in the course are not specific techniques (like jUnit or MockObjects) but instead genuine skills that can apply to many areas of our lives. We could learn those skills from many sources, and this is one of the traditional arguments for a liberal-arts education. When it comes to evaluating courses, conferences, and books, instead of listing specific techniques, we would often be better off listing skills and asking if the course will teach those skills. In that case, "Bug Advocacy" becomes influence, and "How to write a good bug report" becomes writing skills - both of which have universal value to technology workers.

Once we have the skills, the techniques and terminology can help. After all, we the community created test terminology as a servant. Our terminology is a pattern language that allows us to express complex ideas quickly and unambiguously. I may grouse and complain a bit about 'Oracles', 'Heuristics', and 'Context', but when I use the terms, people know what I mean. With this glass/black-box discussion, we the community were becoming a servant of the terminology; we got the essence confused with the accident(**).

Speaking of liberal arts, when I was preparing for the course I asked my wife what books she would recommend to teach systems thinking. Her response was simply "Aristotle".

I would like to know: What color box is that?


//Fin

(*) - http://www.testingeducation.org

(**) - For a great read, please consider Rethinking Systems Analysis and Design by Gerald Weinberg. In the book, Dr. Weinberg describes a teaching method where he has his students analyze the inputs and outputs of a physical black (mechanical) box to determine behavior. It’s a fascinating read, and the skills delivered to his students apply to a whole lot more than BBST.