Tuesday 20 November 2007

XPDay 2007: Fiat Lux: Testing With The Light On - Joan McGalliard, George Joseph & Pushpa Sebastian

Testing team in growing agile project

Teams split according to system functionality rather than business needs

  • No one knew if a story was finished or not

30 devs, 10 testers, 7 BAs, integration team

Embedded testers in development team

At beginning of iteration wrote down stories for iteration

  • each story had page of details: acceptance criteria

Active documentation:

  • tests written in business English
  • runnable
  • italics for input, bold for output — when run, outputs are coloured red or green
  • also had table tests

Documentation is always right since if they fail, the build fails

Tests linked from stories and checked in to source control

The story process:

  1. agreeing acceptance criteria
  2. dev complete
  3. BA/show-and-tell sign off
  4. tester sign off

Can colour code iteration plan to see how far each story has got

Technology involved

Originally used FIT — introduced idea of active documentation

But inflexible, developers didn’t like fixtures

Ended up using Concordion

Tests are written using JUnit but described in an instrumented HTML page

When they fail, they give a stack trace as well as the failures on the resulting page

How to introduce

How to catch up on legacy code?

  • Forget about it — just test for current stories
  • Ended up covering criteria for previous stories too

Having tests ready in time?

  • Works best for 2 testers for every 3 developer pairs and 1 customer/BA

Takes quite a while to write the testing framework & tests

  • 50% of developer time…?

Faulty fixtures

  • Change test data and ensure test fails appropriately
  • Reuse fixtures
  • Pair testers & developers to write the tests

My thoughts: the fixtures are not visible to the customer, and the words in the active documentation do not actually have any connection to the testing code

No comments: