Software Fiction

Jon Bruce continues to improve as a writer. He's got an excellent series going on now called "Startup" - click here, scroll to the bottom of Startup, and read upwards.

In other news ...

I am also in the middle of a back-channel discussion with Ben Simo and Shrini Kulkarni about test frameworks.

This feeds off the idea in my earlier blog post that if your framework makes it hard to test, people won't use it.

Two of the common elements I see in test frameworks are:

(A) Lots of XML
(B) Tough-to-type Syntax

I've explained (A) before, but let me talk for a moment about B.

Often, people have a web application. To test it, they may use framework that drives the browser. The tester then writes test 'code' in a number of possible languages, often Java (Web Driver), Ruby (Watir), or Visual Basic (Quick Test Pro or WinRunner).

The probem is when these frameworks see everything as an object. Yes, that's a problem. Because instead of writing:

SendValueToTag('QTY','100');
PressButton('Submit');
my $val = GetValueAtTheTag('Grand Total');
ASSERT('Grand Total should be fifty bucks',$val, 50.00);

You have to write this:

my $bdy;
$bdy = Object("Browser").Tab(1).page('www.foo.com').html.body;
$body.form('form1').object('tag').label('QTY').setval(100);

$body.object('button').label('submit').push;

my $new_bdy;
$new_bdy = object("Browser").Tab(1).page('www.foo.com').html.body;

my $val;
$val = $new_body.form('form1').object('tag').label('Grand Total').getval();

ASSERT('Grand Total should be fifty bucks',$val, 50.00);

Ah!! Ahhh! Ahhh! My eyes! My eyes!

The sad thing is, I am not exaggerating by much.

So at any given shop, one of a few things happen:

(1) Someone tries to use the framework, and goes through so much pain that they give up.

(2) Someone puts an extreme amount of effort into learning the framework and is actually successful. We'll call him Joe. After that, Joe becomes the in-house expert on the tool. If Joe is assigned to test the software, the software will be tested with that framework. Otherwise, it will probably be tested manually.

(3) You write a custom piece of code that sits on top of the framework that eliminates all the fiddly-bit DOM mappings, so you can just call:

browser.getthetag('qty');
browser.setthetag('foo', 5);

(4) (Hopefully) You find a better framework.

I have quite a few colleagues who have had success with option three. Ruby/Watir, however, looks a lot like #4 - a non-goofy framework that allow you express complex test cases relatively easily, in a language that looks a lot more like English than other options, so it is self-documenting.

My prediction is that the big, slow, dumb tools will continue to dominate the mediocre "no one got fired by buying IBM" space, and smart people will continue to patch them to make them work.

However, if you want to try something completely different - consider learning Ruby, or least watching the Ruby "Switch" Video.