• crash and burn.
  • flip it up on deck.
  • catch it on the first bounce.
  • ping me.
  • you're yankin' my chain.
  • that dog don't hunt.

If you're reading this in the 'States, you probably know what I mean. But if you grew up in the Punjab, and learned your English from an Anglican Church missionary from Hull, those phrases don't mean dick. Which also doesn't mean dick.

I had this driven home in a very dangerous way in the last few weeks.

"Ad hoc testing" is a pretty simple phrase. Ad hoc is Latin for, literally, "to this". Figuratively, I've always thought it meant "impromptu". That's pretty precise, and not subject to interpretation. You'd think.

When we say "ad hoc testing", we mean testing without a script or plan. Trying to break software. Some folks call it "fault method" testing.

One of my QA managers has had his teams doing periodic "ad hoc testing" during the last 5 months of this release. The idea was to run both the test cases we have, and to just kick the tires. When we went into Beta, we found boatloads of bugs that shouldn't have been missed in QA. Hmm. Not good. We sniffed around a little, and discovered a very fundamental communication disconnect.

We said "do ad hoc testing."

They heard "do ad hoc testing."

We thought we meant "do impromptu testing, exercising various divers parts of the product, in mean-spirited ways, to see if you can break the software."

They thought we meant "run test cases randomly selected from the thousand or so test cases we have written, and which we have already run."

We wanted them doing exploratory work, exercising code paths that were unlikely to have been hit thus far in the project. They ended up doing a whole lot of regression testing.

If you know anything about software, you know that these are very different things.

I hate to be trite, but communication is tough. I've turned some of the text above blue, just to visually highlight where I used imprecise language or idiom. It's really hard to write in a precise way, that is not subject to multiple interpretations.

Throw in a bad speaker phone and nine and a half time zones and the problem can be crippling.

I don't have any non-obvious ideas on how to fix this, though I'm still running a background thread on this, and I expect to post more on this later.

No comments: