Tuesday 20 November 2007

XPDay 2007: Agile Alliance ATDD Tools Workshop Report - Antony Marcano

Side issue

Research in Calgary on touch-sensitive table used for agile planning (Frank Maurer)

  • enables card throwing :-)
  • create cards using tablet pc with pen-based input, then appears on shared surface to be moved around


Antony's Blog — things about testing, aggregates other blogs too


Workshop in Portland: discuss cutting-edge advancements in, and envision possibilities for, automated acceptance test driven development tools

  • Tools that help communicate & share with customers to figure out what they want
  • original name was “functional test tools workshop”

Looking at design process using tests

Had a debate: what’s the difference between an example and a test?

  • some took examples and wrapped them, others thought they were different things

Levels of abstraction

  • UCD: Goals, Activities, Tasks
  • Fit tests work better at activity rather than task level
  • Kind of translates to named methods (navigateToJourneyPlanner) vs individual “fill in field with id blah”’


  • Given, when, then
  • behaviour driven development — RSpec (ruby) in process of integrating given, when, then
  • taking hamcrest & literate tests and making a view that allows customers to interact with the tool
    • e.g. like a word processor
  • looked at JDave — didn’t feel as comfortable and natural as JUnit/Hamcrest
  • great for flow tests but not so good for:
    • do this and result in this data (row fixture in Fit)
    • truth tables — have some ideas but too immature right now
  • Nat Pryce refactored tests to be literate, next step to output Word docs from tests
  • Antony would like to enable customer to write tests (but still need conversation)
  • Nat saying that customers need to write tests in conversation with developers, so in code is good
  • thoughts: given, when, then: moving towards Z & specifications — anybody looked at synergy here?

Visualisation of flow

  • Brian Marick blog
    • currently implemented in Ruby, uses Omnigraffle so Mac-only :-)
    • draw a picture and annotate it
    • fear that if pics too detailed, then lots of refactoring needed
  • Ward Cunningham - Eclipse portal (not open source but tightly coupled to Eclipse project)
    • workflow involved in someone being nominated to be a committer on Eclipse
    • after the fact reporting on tests
      • can hover over steps and view form & data particular step
      • also have links to take you to that point in the application with data already set up
    • also used for help documentation — legal dept could see steps involved
    • actual tests captured in text-based format
  • CubicTest — Eclipse plugin to visually design tests with states
    • generates an interactive HTML prototype, based on the tests you have written
  • these tests were captured at task level, but reported at activity level

Model based testing

  • Ben Simo — state oriented model
  • Manually create state & transition model in Excel, and then generate tests from this model
  • agitator — exercises all the paths through the code (uses code as model)
    • more recent tool allows you to generate JUnit tests from suggestions, can then tidy up and rewrite
  • idea — generate a model from the first tests
    • can then generate new tests to explore the software against that model
    • helps find gaps in thinking (wouldn’t be used for auto-generating, more to suggest new tests)

Patterns of self testing software

  • talk of setting up a wiki to start capturing these ideas


  • tools will encourage appropriate level of abstraction
    • avoid refactoring overhead of writing at task level
  • will allow multiple views on tests
  • tightly integrate into documentation generated from application
  • will become more popular to make tests more transparent — available to customers
    • allows customers to provide specific examples
  • Frank Maurer has an open source project called FitClipse
  • Rick Muggridge: zibrieve.com — refactoring Fit tests
  • Green Pepper: using Confluence to write tests (also ties in with Jira)
    • one issue was tying test versions with app versions (may be sorted with Confluence subversion plugin?)

Q&A + Comments

Concordion, decorate an HTML page with tags underneath, and then JUnit runs it

  • concordion.org — open-sourced last month

Profit — supposedly allows transformation between Fit & Java for refactoring

Online discussion: aa-ftt@yahoogroups.com

Jim Shaw/Shore has idea of customer unit tests (not end to end)

  • forcing the customer to think in modular ways

Good thing for a coach to do — encourage always asking questions

  • put on the tester hat and be persistent about asking questions
  • tester is a role that is very useful in any team

Functional tests taking a long time

  • run as separate continuous integration
  • overnight tests did full thing
  • during the day, had a different config that simplified stack and took less time

No comments: