Saturday 3 October 2009

OverTheAir '09: Natural Interaction Gestures

Joe Macleod & Younghee Jung — Nokia Design Team, London

A though-provoking discussion on how interaction with handheld technology develops. Nokia seem to be doing some good research on user interfaces — shame that doesn’t seem to be transferring into the Series60 devices…

Exploring ‘natural’ gesture interactions

  • Not really anything that replaces face to face communication yet
  • Some gestures transfer across cultures well, e.g. fingers on lips: sshhh
  • Some are complicated, e.g. transferring business cards
  • v-sign doesn’t necessarily mean victory now
    • in Asia it is now a default pose for taking photos
    • watch out for reversing it…
      Not quite the default pose for taking photos
  • in China, can count from 1 to 10 on one hand…
    Counting from 1 to 10 in China
  • in Ghana there’s a really sophisticated hand greeting ritual that includes clicking fingers together with the other person
  • research exercise in Nokia to pull together existing gestures used in phones
    • e.g. turn a phone over to silence a ring (Nokia 8800)

Initial research

  • what are natural gestures involving mobile phones?
  • selected 11 different tasks, from checking time to sending something
  • gave people blocks to play with (no buttons)
  • sketched out what people came up with in four cities around the world
  • e.g. silence gestures:
    • stare at it
    • put finger over the speaker
  • had to fight with people to get them to think outside of pressing buttons

Understanding the factors that make gestures work

  • 60 gestures as stimulus mapped to 15 tasks (4 each)
    • some were already known to be not so useful
      Sample gestures used in Nokia gesture research
  • 14 nationalities among participants
  • video recorded each gesture to control how they were presented
  • some issues were physical capability — can I do this? is it comfortable?
    • “I don’t want to feel tired using my phone”
    • “I don’t want to hit someone in a crowded place”
    • “Might I damage my phone?”
  • have loads of examples of bad gestures, e.g. skipping!

Testing with prototypes

  • Nokia Labs built a sensor packed platform to test prototypes

design principles

  • easy to learn
  • easy to perform
    • you will learn it from other people in the real world
  • no unintentional triggers
  • designed for the mobile context
  • scalable visibility
    • some people are more flamboyant than others…
    • “draw a circle” — comes in all sorts of shapes and sizes!
  • fits the task
    • even if the gesture is not from the natural metaphor, it should have some link to the task
  • works together as a language
    • people already mentioning that if there are more than 3 or 4 gestures, they wouldn’t be able to remember… unless they were natural…


  • Isn’t it easier to learn new gestures with a new object?
    • e.g. introducing NFC cards
    • making existing actions a bit smoother
    • introducing new features
  • Do you think you could take inspiration from sign language gestures?
    • Have you ever seen deaf people talking over a video phone…? The language meant to be seen from a distance…
  • Is there anything that allows users to start experimenting with their own gestures? Like users added hashtags and other meta-data to twitter?
    • Some Japanese phones already let you map 4 pre-defined gestures to specific functions
  • Will some gestures inspire a new form factor?

No comments: