OverTheAir '09: Fluid Layouting Techniques for Widget Development

Daniel Herzog

Original slides:

Fluid Layouting Techniques - Over The Air 2009

View more documents from danfooo.

My notes…

• There’s one web — “You don’t want .mobi — it wasn’t a big success”
• widgets should be cross-platform
  • should take a small amount of space on the desktop, but be bigger (full screen) on mobile
• Vodafone widget engine is based on Opera 9

Fonts

• use media queries to switch to high dpi mode, e.g.

    @media all and (-o-touch) {
        a {padding: 1em}
    }

  • uses opera-specific -o-touch
• alternative:

    @media all and (min-resolution: 200dpi) {
        body {font-size: 22px}
    }
    .myDiv {margin: 1em;}

Images

• should use them at native resolution, loaded from server…
• if you’re desperate, use image scaling using screen width percentages

UI-Elements

• use dynamic images, or scalable SVG
• SVG works across most browsers, apart from IE (but may change in 2009)
• Google working on SVG Web — uses Flash plugin to render SVG content
  • But if widget engine doesn’t support SVG, it may not support Flash either
• Daniel uses Illustrator to create initial SVG, then uses TextMate to hand edit it…
  • Can also use Inkscape but the UI is horrible

More resolutions…

• Portrait & landscape, docked mode (also in portrait and landscape!)
• Deal with these using window.resizeTo
• Call this initially and when the orientation changes

Future

• We’re in trouble — widget runtimes need to return real dpi/ppi values
• Not possible at the moment since developers assume that 10pt is a certain number of pixels
• Would be nice to have an extra CSS property for the real dpi

And another thing… zooming widgets

• Zooming enlarges pixels — see Opera for an example
• But SVG still scales effectively
• Another example: WidgetCity is a widget version of Sim City!

OverTheAir '09: Natural Interaction Gestures

Joe Macleod & Younghee Jung — Nokia Design Team, London

A though-provoking discussion on how interaction with handheld technology develops. Nokia seem to be doing some good research on user interfaces — shame that doesn’t seem to be transferring into the Series60 devices…

Exploring ‘natural’ gesture interactions

• Not really anything that replaces face to face communication yet
• Some gestures transfer across cultures well, e.g. fingers on lips: sshhh
• Some are complicated, e.g. transferring business cards
• v-sign doesn’t necessarily mean victory now
  • in Asia it is now a default pose for taking photos
  • watch out for reversing it…
    
• in China, can count from 1 to 10 on one hand…
  
• in Ghana there’s a really sophisticated hand greeting ritual that includes clicking fingers together with the other person
• research exercise in Nokia to pull together existing gestures used in phones
  • e.g. turn a phone over to silence a ring (Nokia 8800)

Initial research

• what are natural gestures involving mobile phones?
• selected 11 different tasks, from checking time to sending something
• gave people blocks to play with (no buttons)
• sketched out what people came up with in four cities around the world
• e.g. silence gestures:
  • stare at it
  • put finger over the speaker
• had to fight with people to get them to think outside of pressing buttons

Understanding the factors that make gestures work

• 60 gestures as stimulus mapped to 15 tasks (4 each)
  • some were already known to be not so useful
    
• 14 nationalities among participants
• video recorded each gesture to control how they were presented
• some issues were physical capability — can I do this? is it comfortable?
  • “I don’t want to feel tired using my phone”
  • “I don’t want to hit someone in a crowded place”
  • “Might I damage my phone?”
• have loads of examples of bad gestures, e.g. skipping!

Testing with prototypes

• Nokia Labs built a sensor packed platform to test prototypes

design principles

• easy to learn
• easy to perform
  • you will learn it from other people in the real world
• no unintentional triggers
• designed for the mobile context
• scalable visibility
  • some people are more flamboyant than others…
  • “draw a circle” — comes in all sorts of shapes and sizes!
• fits the task
  • even if the gesture is not from the natural metaphor, it should have some link to the task
• works together as a language
  • people already mentioning that if there are more than 3 or 4 gestures, they wouldn’t be able to remember… unless they were natural…

Q&A

• Isn’t it easier to learn new gestures with a new object?
  • e.g. introducing NFC cards
  • making existing actions a bit smoother
  • introducing new features
• Do you think you could take inspiration from sign language gestures?
  • Have you ever seen deaf people talking over a video phone…? The language meant to be seen from a distance…
• Is there anything that allows users to start experimenting with their own gestures? Like users added hashtags and other meta-data to twitter?
  • Some Japanese phones already let you map 4 pre-defined gestures to specific functions
• Will some gestures inspire a new form factor?

OverTheAir '09: Mobile Visual Interactivity

Geoff Ballinger — mobile acuity

Geoff runs a mobile development agency in Scotland, specialising in applying vision technologies.

• lots of real people use the camera on their phone
• in 16-34 age group 78% have sent an MMS, 48% cameras regularly
• there are graveyards in Japan with QR codes on each headstone!
  • scan them to get some details about the person
• examples:
  • take a picture of a goal and MMS it — it responds with a mobile web page that tells you whether you scored, then pulls you in to further interaction
  • Nike PhotoiD — find some colours you like, send it in and get a picture of trainers coloured that way
• Visual search apps on iPhone store (CDs and books):
  • Snaptell Explorer from Amazon
  • Barnes and Noble store app
  • Mobile Acuity Visual Shopper — not yet released
• recognise images from database of 10m
• code libraries
  • Haar recogniser
  • ARToolkit
• web service APIs: Kooaba & Mobile Acuity (available on a per-use basis)
  • feature recognition from images

OverTheAir '09: Modelling the User Experience

Bryan Rieger — yiibu

Bryan comes from a theatre and animation background and was sorely disappointed by the tools available for designing interactive services. He and his partner Stephanie have developed a lightweight prototyping mechanism for interactive mobile services. I like the idea — especially the bit about throwing away lots of ideas in order to explore.

Tools for traditional animation

• scripts & storyboards —> models & layouts
• adding structure, but always changing
• dope sheets, now known as timelines
• pencil tests
• animatics — complete but not finished
  • stills set to music/voice over, possibly panned and scanned

Tools for Mobile design

• wireframes and… more wireframes
• what are they?
  • layout, behaviour and flow all in one document
• are we expecting too much from one document?
• Bryan started dropping little nonsense text on the descriptions of wireframes to see if anyone was reading them — turns out nobody was!
• documentation gets bigger and bigger but not readable
• document management != design

Prototypes

• where do you start and where do you stop
• paper -> HTML -> Flash -> Python, Java, …
  • the more you get to development, the more people don’t want to tear it down
  • developers (and those who manage them) should learn the value of the spike — a throwaway prototype that you really do throw away
• want to get something more concrete than paper but has limitations
  • really difficult to share
  • usually requires some mediation

Agile

• iterations are good for design
• you can explore more, develop new different ideas and throw away bad ones
• how can I make lots of mistakes? how can I get visibility of the project as a whole?

Integrating knowledge from animation

• applying animatics to wireframes
  • want something interactive, but still sketched
  • and the sketched part must include the flow and behaviour as well as the layout
• disposable data models:
  • views, states and events
  • views and states are scanned in from paper prototypes
  • events switch from state to state or to different views
  • creating these is relatively easy, esp. using Fireworks as a tool
  • Fireworks allows you to create multiple states side by side, all based on a single view
• can switch in different JPEGs really easily — iterate visual design
• can easily alter model (flow & behaviour) by changing XML
  • at the moment, the XML is hand-built
• player built using Mobile Processing, so runs on real devices (as JavaME midlet)
  • same player for any design — just load and go
• models (including graphics) take about 1 day to create
  • so can create 60+ models over several weeks
• is this system similar to Cascade Breeze, or PhoneGap?

Experience

• created lots of documentation up front, but then started culling pages
• shipped prototypes with the documentation
• created 60+ models — allowed change
• took 2 designers 9 weeks of work
• led to clearer wireframes
  • but didn’t replace them — this is an additional tool
• much greater user testing since lots of people could use it
• slowly refining the tool:
  • adding gestures and touch interactivity
• don’t model the whole app — just specific flows