Interview?

Software Quality Engineering recently sent me a bunch of inteview questions for a StickToolLook e-newsletter. Writing a reply was hard, as I wanted to provide real information that was worth reading, not just pontificate or repeat the same old thing. My replies are below; I'm interested in what you think.


SQE: What are the benefits of a tool developed for an entire market? What about a tool designed to focus on a particular need?

That's a really tricky question, because within every market there is a smaller market. If we start with tools designed for everyone who ever touches software development, we can find tools inside of that. Say, tools for just testers – then just performance testers. Inside of that market are tools for web performance testers, java testers, and database testers. Inside of that we have tools for each specific flavor of database.

Overall, I like tools as niche as a I can get them, because the vendor can solve a specific problem and solve it well. The higher-level tools offer some level of integration and file sharing, and workflow that just can’t exist in a specific tool. Also, the more general a tool, the better chance there will be to find a user group, consultant, or support when things go bad.

SQE: What are the cons to using either type of tool?

Well, for example, if you use a general tool for all databases, calling your sales rep and ask about Oracle support, and he replies "yeah, there’s a driver for that." That’s bad news. At best, you’re going to muck about for a day or three before finally getting the software to mostly work. At worst, it'll go back on the shelf. General tools generally have a main market; like ordering the fish in a steakhouse, you never know quite what you’re gonna get.

I have heard of a few general tools that work in lots of environments; McCabe has amazing support and I've heard good things about Worksoft certify. You just have to be certain to check a vendors claims carefully – yourself.

Of course, if you use the specialized, Database-Specific tool and write custom code for it, then when you convert environments you might have to throw all that work away and start over.

SQE: Is it possible to develop a tool or a tool suite that can be all things to all people? Or even all things to one person?

I think it’s a matter of philosophy. UNIX, for example, is a collection of small tools that can be tied together in really interesting ways to do just about anything; I think the "Swiss Army Chainsaw" is my favorite term to describe it. Is UNIX a single tool or a collection of tools? (Grins) Who Cares?

There are certainly performance testing tools, record/playback tools, and development environments people spend all day in, and some of them have very passionate and active users. My colleague Mike Kelly is fond of the phrase "Role Players", to mean that the person likes to perform a function and do it well. If someone wants to be an excellent performance tester for websites, then sure, that person could live in one tool, align himself with that vendor, and probably make a very good living as an employee or consultant. That just isn't me, and you never know when a specialty is going to be destroyed – say, for example, when websites start to support wireless devices and the bottleneck for performance shifts.


SQE: On the other hand, can a tool be too specific?

Certainly! I don’t know anyone who uses awk or sed anymore, for example. Those were great little programming languages, but perl cam along and offered all the exact same features plus more.

Of course, if the tool is open source, it doesn’t really matter if the tool is too specific, because it's free, and you can put it away and pull it out if and when you have the need again. If it’s for-purchase, the market will sort it out – the vendor will either expand the product or fail in the marketplace. (You have to love capitalism, after all.)

SQE: Which type of tool is more prevalent in the market right now?

I think vendor-supported tools tend to go big and open source tools tend to be small. The open source tools are often developed by a specific person to solve his own problems, and growing the tool won't make him any money. So open-source tools grow through some other guy adding features that he needs. For-purchase tools are driven by companies that want to make money; they have an economic reason to try to reach as many people as possible. I hear a lot of good things about Visual Studio Team System, which is big, big, big general purpose tool. I think the market is still figuring itself out, but right now most of the new tools I see are specific tools for a small market, created by small companies who have expertise in that area – and I think that is very good thing.

SQE: Where do you see the market headed in the future?

I think software testing is an immature field, and software quality even less so. That means we need to try a bunch of different approaches, see what works, and then build up tools around that. Like I mentioned earlier, I see little companies doing just that. If they succeed, economic forces will expand those products a little bit, while hopefully the "big" products will get out of the clouds and come down to the real world a little bit more. Eventually, I predict a fight for the middle.