XPDay 2007: Agile Alliance ATDD Tools Workshop Report - Antony Marcano

Side issue

Research in Calgary on touch-sensitive table used for agile planning (Frank Maurer)

• enables card throwing :-)
• create cards using tablet pc with pen-based input, then appears on shared surface to be moved around

Intro

Antony's Blog — things about testing, aggregates other blogs too

Report

Workshop in Portland: discuss cutting-edge advancements in, and envision possibilities for, automated acceptance test driven development tools

• Tools that help communicate & share with customers to figure out what they want
• original name was “functional test tools workshop”

Looking at design process using tests

Had a debate: what’s the difference between an example and a test?

• some took examples and wrapped them, others thought they were different things

Levels of abstraction

• UCD: Goals, Activities, Tasks
• Fit tests work better at activity rather than task level
• Kind of translates to named methods (navigateToJourneyPlanner) vs individual “fill in field with id blah”’

Vocabulary

• Given, when, then
• behaviour driven development — RSpec (ruby) in process of integrating given, when, then
• taking hamcrest & literate tests and making a view that allows customers to interact with the tool
  • e.g. like a word processor
• looked at JDave — didn’t feel as comfortable and natural as JUnit/Hamcrest
• great for flow tests but not so good for:
  • do this and result in this data (row fixture in Fit)
  • truth tables — have some ideas but too immature right now
• Nat Pryce refactored tests to be literate, next step to output Word docs from tests
• Antony would like to enable customer to write tests (but still need conversation)
• Nat saying that customers need to write tests in conversation with developers, so in code is good
• thoughts: given, when, then: moving towards Z & specifications — anybody looked at synergy here?

Visualisation of flow

• Brian Marick blog
  • currently implemented in Ruby, uses Omnigraffle so Mac-only :-)
  • draw a picture and annotate it
  • fear that if pics too detailed, then lots of refactoring needed
• Ward Cunningham - Eclipse portal (not open source but tightly coupled to Eclipse project)
  • workflow involved in someone being nominated to be a committer on Eclipse
  • after the fact reporting on tests
    • can hover over steps and view form & data particular step
    • also have links to take you to that point in the application with data already set up
  • also used for help documentation — legal dept could see steps involved
  • actual tests captured in text-based format
• CubicTest — Eclipse plugin to visually design tests with states
  • generates an interactive HTML prototype, based on the tests you have written
• these tests were captured at task level, but reported at activity level

Model based testing

• Ben Simo — state oriented model
• Manually create state & transition model in Excel, and then generate tests from this model
• agitator — exercises all the paths through the code (uses code as model)
  • more recent tool allows you to generate JUnit tests from suggestions, can then tidy up and rewrite
• idea — generate a model from the first tests
  • can then generate new tests to explore the software against that model
  • helps find gaps in thinking (wouldn’t be used for auto-generating, more to suggest new tests)

Patterns of self testing software

• talk of setting up a wiki to start capturing these ideas

Conclusions

• tools will encourage appropriate level of abstraction
  • avoid refactoring overhead of writing at task level
• will allow multiple views on tests
• tightly integrate into documentation generated from application
• will become more popular to make tests more transparent — available to customers
  • allows customers to provide specific examples
• Frank Maurer has an open source project called FitClipse
• Rick Muggridge: zibrieve.com — refactoring Fit tests
• Green Pepper: using Confluence to write tests (also ties in with Jira)
  • one issue was tying test versions with app versions (may be sorted with Confluence subversion plugin?)

Q&A + Comments

Concordion, decorate an HTML page with tags underneath, and then JUnit runs it

• concordion.org — open-sourced last month

Profit — supposedly allows transformation between Fit & Java for refactoring

Online discussion: aa-ftt@yahoogroups.com

Jim Shaw/Shore has idea of customer unit tests (not end to end)

• forcing the customer to think in modular ways

Good thing for a coach to do — encourage always asking questions

• put on the tester hat and be persistent about asking questions
• tester is a role that is very useful in any team

Functional tests taking a long time

• run as separate continuous integration
• overnight tests did full thing
• during the day, had a different config that simplified stack and took less time