Saturday 3 October 2009

OverTheAir '09: Fluid Layouting Techniques for Widget Development

Daniel Herzog

Original slides:

My notes…

  • There’s one web — “You don’t want .mobi — it wasn’t a big success”
  • widgets should be cross-platform
    • should take a small amount of space on the desktop, but be bigger (full screen) on mobile
  • Vodafone widget engine is based on Opera 9


  • use media queries to switch to high dpi mode, e.g.
      @media all and (-o-touch) {
          a {padding: 1em}
    • uses opera-specific -o-touch
  • alternative:
      @media all and (min-resolution: 200dpi) {
          body {font-size: 22px}
      .myDiv {margin: 1em;}


  • should use them at native resolution, loaded from server…
  • if you’re desperate, use image scaling using screen width percentages


  • use dynamic images, or scalable SVG
  • SVG works across most browsers, apart from IE (but may change in 2009)
  • Google working on SVG Web — uses Flash plugin to render SVG content
    • But if widget engine doesn’t support SVG, it may not support Flash either
  • Daniel uses Illustrator to create initial SVG, then uses TextMate to hand edit it…
    • Can also use Inkscape but the UI is horrible

More resolutions…

  • Portrait & landscape, docked mode (also in portrait and landscape!)
  • Deal with these using window.resizeTo
  • Call this initially and when the orientation changes


  • We’re in trouble — widget runtimes need to return real dpi/ppi values
  • Not possible at the moment since developers assume that 10pt is a certain number of pixels
  • Would be nice to have an extra CSS property for the real dpi

And another thing… zooming widgets

  • Zooming enlarges pixels — see Opera for an example
  • But SVG still scales effectively
  • Another example: WidgetCity is a widget version of Sim City!

OverTheAir '09: Natural Interaction Gestures

Joe Macleod & Younghee Jung — Nokia Design Team, London

A though-provoking discussion on how interaction with handheld technology develops. Nokia seem to be doing some good research on user interfaces — shame that doesn’t seem to be transferring into the Series60 devices…

Exploring ‘natural’ gesture interactions

  • Not really anything that replaces face to face communication yet
  • Some gestures transfer across cultures well, e.g. fingers on lips: sshhh
  • Some are complicated, e.g. transferring business cards
  • v-sign doesn’t necessarily mean victory now
    • in Asia it is now a default pose for taking photos
    • watch out for reversing it…
      Not quite the default pose for taking photos
  • in China, can count from 1 to 10 on one hand…
    Counting from 1 to 10 in China
  • in Ghana there’s a really sophisticated hand greeting ritual that includes clicking fingers together with the other person
  • research exercise in Nokia to pull together existing gestures used in phones
    • e.g. turn a phone over to silence a ring (Nokia 8800)

Initial research

  • what are natural gestures involving mobile phones?
  • selected 11 different tasks, from checking time to sending something
  • gave people blocks to play with (no buttons)
  • sketched out what people came up with in four cities around the world
  • e.g. silence gestures:
    • stare at it
    • put finger over the speaker
  • had to fight with people to get them to think outside of pressing buttons

Understanding the factors that make gestures work

  • 60 gestures as stimulus mapped to 15 tasks (4 each)
    • some were already known to be not so useful
      Sample gestures used in Nokia gesture research
  • 14 nationalities among participants
  • video recorded each gesture to control how they were presented
  • some issues were physical capability — can I do this? is it comfortable?
    • “I don’t want to feel tired using my phone”
    • “I don’t want to hit someone in a crowded place”
    • “Might I damage my phone?”
  • have loads of examples of bad gestures, e.g. skipping!

Testing with prototypes

  • Nokia Labs built a sensor packed platform to test prototypes

design principles

  • easy to learn
  • easy to perform
    • you will learn it from other people in the real world
  • no unintentional triggers
  • designed for the mobile context
  • scalable visibility
    • some people are more flamboyant than others…
    • “draw a circle” — comes in all sorts of shapes and sizes!
  • fits the task
    • even if the gesture is not from the natural metaphor, it should have some link to the task
  • works together as a language
    • people already mentioning that if there are more than 3 or 4 gestures, they wouldn’t be able to remember… unless they were natural…


  • Isn’t it easier to learn new gestures with a new object?
    • e.g. introducing NFC cards
    • making existing actions a bit smoother
    • introducing new features
  • Do you think you could take inspiration from sign language gestures?
    • Have you ever seen deaf people talking over a video phone…? The language meant to be seen from a distance…
  • Is there anything that allows users to start experimenting with their own gestures? Like users added hashtags and other meta-data to twitter?
    • Some Japanese phones already let you map 4 pre-defined gestures to specific functions
  • Will some gestures inspire a new form factor?

Tuesday 29 September 2009

OverTheAir '09: Mobile Visual Interactivity

Geoff Ballinger — mobile acuity

Geoff runs a mobile development agency in Scotland, specialising in applying vision technologies.

  • lots of real people use the camera on their phone
  • in 16-34 age group 78% have sent an MMS, 48% cameras regularly
  • there are graveyards in Japan with QR codes on each headstone!
    • scan them to get some details about the person
  • examples:
    • take a picture of a goal and MMS it — it responds with a mobile web page that tells you whether you scored, then pulls you in to further interaction
    • Nike PhotoiD — find some colours you like, send it in and get a picture of trainers coloured that way
  • Visual search apps on iPhone store (CDs and books):
  • recognise images from database of 10m
  • code libraries
    • Haar recogniser
    • ARToolkit
  • web service APIs: Kooaba & Mobile Acuity (available on a per-use basis)
    • feature recognition from images

OverTheAir '09: Modelling the User Experience

Bryan Rieger — yiibu

Bryan comes from a theatre and animation background and was sorely disappointed by the tools available for designing interactive services. He and his partner Stephanie have developed a lightweight prototyping mechanism for interactive mobile services. I like the idea — especially the bit about throwing away lots of ideas in order to explore.

Tools for traditional animation

  • scripts & storyboards —> models & layouts
  • adding structure, but always changing
  • dope sheets, now known as timelines
  • pencil tests
  • animatics — complete but not finished
    • stills set to music/voice over, possibly panned and scanned

Tools for Mobile design

  • wireframes and… more wireframes
  • what are they?
    • layout, behaviour and flow all in one document
  • are we expecting too much from one document?
  • Bryan started dropping little nonsense text on the descriptions of wireframes to see if anyone was reading them — turns out nobody was!
  • documentation gets bigger and bigger but not readable
  • document management != design


  • where do you start and where do you stop
  • paper -> HTML -> Flash -> Python, Java, …
    • the more you get to development, the more people don’t want to tear it down
    • developers (and those who manage them) should learn the value of the spike — a throwaway prototype that you really do throw away
  • want to get something more concrete than paper but has limitations
    • really difficult to share
    • usually requires some mediation


  • iterations are good for design
  • you can explore more, develop new different ideas and throw away bad ones
  • how can I make lots of mistakes? how can I get visibility of the project as a whole?

Integrating knowledge from animation

  • applying animatics to wireframes
    • want something interactive, but still sketched
    • and the sketched part must include the flow and behaviour as well as the layout
  • disposable data models:
    • views, states and events
    • views and states are scanned in from paper prototypes
    • events switch from state to state or to different views
    • creating these is relatively easy, esp. using Fireworks as a tool
    • Fireworks allows you to create multiple states side by side, all based on a single view
  • can switch in different JPEGs really easily — iterate visual design
  • can easily alter model (flow & behaviour) by changing XML
    • at the moment, the XML is hand-built
  • player built using Mobile Processing, so runs on real devices (as JavaME midlet)
    • same player for any design — just load and go
  • models (including graphics) take about 1 day to create
    • so can create 60+ models over several weeks
  • is this system similar to Cascade Breeze, or PhoneGap?


  • created lots of documentation up front, but then started culling pages
  • shipped prototypes with the documentation
  • created 60+ models — allowed change
  • took 2 designers 9 weeks of work
  • led to clearer wireframes
    • but didn’t replace them — this is an additional tool
  • much greater user testing since lots of people could use it
  • slowly refining the tool:
    • adding gestures and touch interactivity
  • don’t model the whole app — just specific flows