Thursday 12 November 2009

Apple iPhone Tech Talk London 2009: Overview

Apparently I can blog about this year’s Apple iPhone Tech Talk World Tour, so here’s my notes.

This was a much larger event than last year — and I wasn’t the only person surprised by how many developers turned up. Since the registration process involved entering an iTunes URL, most of the people present must have written an app and got it into the store. However, the gender balance was even worse some recent developer events I’ve attended — it looked like there were more women on the Apple team than in the whole audience…

The kickoff session was a quick rundown of where iPhone development has got to since it launched only 15 months ago (seems like longer, possibly because the jailbreakers got there a fair amount earlier…), and the rest of the day was spent going into details (I'll cover the sessions I attended in later posts). I really hope the Apple guys put their presentations up soon, as there was a lot of great detail that I couldn’t type fast enough to get down :-)

stats

  • only 15 months since App Store launched (though an extra year of iPhone)
    • 2 Billion apps, means 180,000 apps an hour…
    • 100,000 apps in app store
  • more than 50 million devices
  • 81 countries
  • 125,000 registered developers
    • so at least 25,000 who paid money but haven’t put anything in the store yet!

timeline

  • jun 08: seeding SDK & emulator to all devs
  • jul 08: app store live
  • sep 08: customer reviews only by purchasers & versions
  • oct 08: crash reporter & logs
  • mar 09: developer discussion forum (still beta), more customer service agents
  • jun 09: quick reference guides
  • jul 09: news & announcements + better developer support
  • sep 09: new app store categories (genius, essentials), top grossing lists, app store resource centre
  • oct 09: in-app purchase available to free apps

the best apps are…

  • simple or sophisticated (hiding complexity)
  • lightweight
  • intuitive
  • small pieces loosely joined

iPhone users have changed

  • not necessarily technically savvy
  • bringing technology into new places

specific examples of innovative apps

  • each app has 5-7 seconds to sell itself to the user
    • it helps to have a memorable aspect
  • use the latest technologies
  • go the extra mile
  • keep it fresh

  • urbanspoon

    • shake to randomly choose a restaurant in the style of a fruit machine
  • peaks
    • Augmented Reality (AR) for mountains
  • car finder
    • AR for finding car
  • zipcar
    • book and find cars
    • unlock the car over network, when you’re close to it
  • snaptell
    • picture identification
  • postage ~ postcards
    • take a picture and send an electronic postcard
    • can apply simple visual effects
    • visually interesting error messages (look like envelopes with postmarks)
  • timetuner
    • bedside radio
    • location aware — will take timezone into account
    • can set up custom alerts — different radio stations on different days
    • simple sleep timer on main screen
  • weightbot
    • way over the top UI polish…
    • real sounds
    • makes it fun to enter your weight
  • tweetie 2
    • only presents UI when you need it
    • swipe down to refresh at top
    • swipe across a tweet to see options
  • guitartoolkit
    • swipe across to play strings for a selected chord
    • has tuner
    • has metronome that keeps playing when screen goes dark
    • can do this by setting app type — determines what happens when
  • 1112
    • exploration game with lots of extra touches
  • ocarina
    • with worldwide
    • and sonic lighter — added messages for choice of flame to mean something
  • roambi
    • data interaction with data from SAP, Salesforce etc.
    • lots of visual interaction with data
    • pull through folders, seeing preview of front one
    • pie chart can swivel round to see details of the one on the bottom
    • charts can
  • things
    • auto synchronized when iPhone comes into same WiFi network
  • MLB.com At Bat
  • Flick Fishing
    • using GameKit to hook up to other iPhones
  • myStarbucks
    • can send your standard order to someone else so they can order for you
    • (not currently available in UK store)
  • ramp champ
    • in-app purchases
    • shows preview, details and price
    • very good catalogue implementation
  • iceberg reader
    • iTunes puts up a confirmation dialog of purchase as well
    • shows progress of download in-app
    • (again, not yet available in the UK store — though individual books are)

Apple iPhone Tech Talk London 2009: Networking: From Sockets to GameKit

Paul Danbold danbold@apple.com — Evangelist

This was another really fast presentation with loads of detail. I look forward to grabbing the presentation from the iPhone dev site.

  • don’t want to use CFSocket or BSD Socket
    • don’t let you bring up wireless, or involve security
  • service discovery:
    • NSNetServices, CFNetServices, dns-sd, BSD Sockets
    • dns-sd is Bonjour or Zeroconf

URL Loading system

  • NSURLRequest, NSURLConnection & about 10 other classes (also mutable ones)
  • there’s sample code for validating URLs
  • recommend to use asynchronous API style:
    • use delegate mechanism — calls connection:didReceiveResponse:
  • also connection:didReceiveAuthenticationChallenge:
  • finally connectionDidFinishLoading:
  • default behaviours should work fine

Local area networking — service discovery

  • kick off NSNetServiceBrowser in asynchronous mode, w/delegation
  • browser searchForServicesOfType:inDomain:
    • e.g. type _http._tcp in domain local. — don’t go beyond any routers
    • can look for your own types too such as _foo._tcp
  • then get myServiceBrowser:didFindService:moreComing:
  • call NSNetworkService getInputStream:outputStream to resolve
    • this takes time (many seconds)
    • wait for the user to choose before resolving
    • and let the user decide when to cancel
    • resolveWithTimeout:0.0
  • must release input & output stream objects returned by NSNetService
    • bug in Apple’s code — doesn’t obey normal Cocoa behaviour

OutputStream & InputStream

  • NSStreamEventHasSpaceAvailable is probably the interesting one
  • though you want to handle the errors too…

advertising & publishing

  • advertise service with NSNetService initWithDomain:type:name:port
    • domain can be empty — means local
    • name can be blank too — will use device’s iTunes name
  • then call publish on service

stop when you’re done

  • browsing is fairly lightweight on the network, but you should stop anyway
  • the same goes for publishing, when you’ve got all your connected clients

register types & ports

game kit — peer to peer

  • 3.1 adds WiFi support as well as Bluetooth
  • GKSession initWithSessionID:displayName:sessionMode
    • session ID is your Bonjour service type — how you advertise yourself
    • again, can leave displayName as nil for iTunes name
    • sessionMode can be peer to peer (1 to 1) or client/server (multiple clients)
  • can’t physically go over 10-20 clients with Bluetooth
    • 3.1 and later is a lot better, but still no more than 3-4 clients
    • Bluetooth also has low bandwidth
  • set up delegate and set available to YES
  • check other peers with peersWithConnectionState:
    • can be available, already connected, in the process of getting connected
  • connectToPeer:withTimeout:
    • generally set timeout to zero to let user cancel if they want
  • actual connection made with session:didReceiveConnectionRequestFromPeer: and acceptConnectionFromPeer:error:
  • must monitor the network using session:peer:didChangeState:
    • other players can drop out, so you’d have to tear down your session
  • send data with sendDataToAllPeers:withDataMode:error: and sendDataToPeers:...
    • can choose unreliable (UDP) or reliable (TCP) data modes
    • as always TCP imposes a performance cost so use sparingly
  • receive data with setDataReceiveHandler:withContext: and receiveData:fromPeer:inSession:context:
  • always check for errors — buffers fill up on low bandwidth networks
    • check it by using a slow network
  • when you’re finished, tear down with disconnectFromAllPeers:
  • there’s a GKPeerPickerController for setting up Bluetooth peers
    • doesn’t work for client/server or WiFi
  • the only way to find out if Bluetooth is turned on is to use the GKPeerPickerController (it uses private APIs…)
  • all data sent & received is NSData-wrapped — design your packaging for network efficiency first and coding efficiency second

Voice chat

  • GKVoiceChatClient and ~Service
  • sets up socket interface
  • handles microphone, echo suppression, etc

network challenges

scoped routing and reachability

  • iphone 3.0 and later provides scoped routing
    • keeps both cellular and wifi up at the same time — so cellular connections keep going when you enter a wifi hotspot
    • but this puts a toll on the battery, so you should behave and check for networking changes
  • use SystemConfiguration APIs to monitor network state
    • Am I on the network and what kind of network is it?
    • cannot tell you that packets will arrive…
    • if network changes, finish what you’re doing, close connection and start a new one
  • use SCNetworkReachabilityRef and check for flags
    • but again use callback APIs since the calls take about 30 seconds
  • no point using this to pre-flight as things may change
    • just use to monitor
  • can set UIRequiresPersistentWiFi key
    • keeps WiFi live while your app is live
    • shows up UI elements automatically (like offering to turn off Airplane mode)
    • but if the device sleeps, you still lose your WiFi connection

battery use

  • listening is cheap, transmitting is expensive
  • 3GS sending leaves the antenna in high powered state for 5 seconds
    • so don’t send every 5 seconds — it will leave it in high power continuously
  • compress data, send large chunks
  • tear down connections when reachability changes

other tips

  • try to use the cache as much as possible
  • try to resume downloads rather than restart
  • use pipelining if poss.
    • ask for lots of things at once and be careful about return order
    • some web proxies and apache configs just break pipelining
  • also use multiple connections
  • but different network conditions and different servers require different tuning
  • test in areas where you know the network is flaky!
  • use a packet analyzer — see QA1176
  • try to isolate the user from network problems
    • often there’s a transient glitch — try again before alerting the user
  • see also Technote TN2152
  • look for developer forum postings by eskimo1 :-)

there’s some new sample code available:

  • SimpleNetworkStreams
  • SimpleURLConnection
  • SimpleFTPSample
  • Reachability — there’s a new version! The old version had bad code :-)
  • BonjourWeb
  • WiTap — peer to peer, but doesn’t use GameKit
  • GKTank
  • GKRocket — work in progress, with voice chat too

Apple iPhone Tech Talk London 2009: Testing and Debugging Your iPhone Application

Just before this session, I had an interesting discussion with Mike Llewellyn, author of Broadersheet, about unit testing. He uses defensive code with asserts and logging and doesn't think there is enough value in unit testing for mostly UI code.

However, Apple have converted to using unit tests and are making it easy for XCode developers to take advantage of them.

At Kizoom, we’ve used unit tests to test our controller code — ensuring that given a particular input, it goes to the right result view with the appropriate model. This is made easier by using three20 — the result view is identified by a string rather than an object (though using OCMock would let you assert about objects too).

Here’s what Michael Jurewitz jurewitz@apple.com — App Frameworks & Dev Tools Evangelist had to say:

  • thousands of tests in Core Data
    • roughly 90% coverage (that’s very impressive — it’s hard to maintain more than 80%)
    • fully test driven from the start
  • similarly large amount of tests in WebKit
  • Application tests get injected into application and can drive running app

static analysis

  • has found and fixed thousands of bugs in Snow Leopard & iPhone OS :-)
  • use it to learn and enforce good Cocoa practices
  • detects dead stores: unused variables
  • see “Run Static Analyzer” in Project properties / Build
    • might not want to have this for all configurations
    • gets picked up by xcodebuild on the command line

organizer

  • check the console if you get a crash
  • can download app’s data from actual device:
    • check Applications under Summary — your app will have a disclosure arrow

beta testing

  • create a distribution certificate in the iPhone portal
  • create an adhoc provisioning profile
    • can still use a wildcard identifier
  • load it up into XCode
  • make a beta testing config
    • duplicate the release config
    • set Code Signing Entitlements to Entitlements.plist (a new file)
    • use appropriate provisioning profile
    • create new Entitlements: set get-task-allow to false
    • disallows debugging
    • if you want people to debug, then send them a debug build…
  • build + send zip of build + provisioning profile to beta testers
  • they can unzip & drag both to iTunes

crashes

  • can get crash log from user — iTunes syncs them
    • Mac & Windows store them in different places, but can find…
    • also available in iTunes Connect
  • can get Organizer to symbolicate crash log for you
    • your .app and .dSym must be locatable by Spotlight — so keep them

low memory logs

  • see count of pages: each page is 4Kb
  • SpringBoard shares memory with your app — it handles presentation on screen
  • mediaserverd is iPod
  • if you find that something other than your app or SpringBoard is the largest memory user, then file a bug…

other crashes

EXC_CRASH (SIGABRT)

  • 0xdeadfa11 — termination due to too much memory
  • 0x8badf00d — watchdog timeout

EXC_BAD_ACCESS (SIGBUS)

  • objc_msgSend — happens when you message a deallocated object

instruments

  • Zombies template — record reference counts & enabled NSZombie detection
    • see extended detail view using rectangle icon at bottom of window
    • shows stack trace for each event
    • can jump through to code view, then jump into XCode editor

Apple iPhone Tech Talk London 2009: Integrating Web Content into iPhone Apps

Vicki Murley — Safari Technologies Evangelist vicki@apple.com

  • iTunes Music Store home page & GarageBand lessons are both HTML-based — included into mac apps
  • UIWebKit is not Safari — doesn’t support new windows & tabs + different memory management

Loading local resources

  • webview load: baseUrl should be initial path so that relative URLs work
  • can’t refer to local resources in remote content
    • but can have overrides in local resources

Performance optimisation

  • minimise the total number of UIWebViews
    • use AJAX
    • build views offscreen then animate them in, and delete unused
    • see examples in DashCode
  • no explicit cache control, so can’t free up memory easily
    • but can set javascript objects to null

Authentication

  • can embed in URL, e.g. http://user:password@www.example.com
  • to prompt the user, use NSUrl APIs
    • get didReceiveAuthenticationChallenge
    • can persist credentials for session

Javascript to Objective-C and back

  • evaluating javascript in Objective-C:
    • try to keep javascript simple — just a function call to a separate file
    • make sure .js files are copied, not compiled
  • can go the other way round too:
    • use Javascript to change to a custom URL (still http:// though)
    • use webview delegate: shouldStartLoadWithRequest
    • return NO to say stop processing

using images more effectively

  • be aware that file size is not memory size — especially for JPEGs
    • in memory, images are stored as pixel by pixel values — no compression at all
  • use generated images through CSS
    • CSS gradients — they support transparency too
    • CSS masks — properties analogous to background
    • can use gradients or SVG images for masks
    • CSS reflections: -webkit-box-reflect
    • again, use gradient as final parameter
  • iPhone uses Helvetica Neue for standard interface
  • make buttons using CSS:
    • round corners: 10px is about right
    • add slight shadow behind text: text-shadown: 0px 0px 8px black;
    • border at bottom: -webkit-box-shadow: 0px 2px gray;
    • white overlaying gradient
    • disable standard text behaviour:
      • -webkit-tap-highlight-color: transparent
      • -webkit-touch-callout: none
      • -webkit-user-select: none
    • also make it behave like a button when you press it
    • have an enabled and disabled backgroundColor using onTouch
  • see DashCode for other examples

  • even if you have larger resources in total, having fewer separate resources can make the page load faster

CSS transitions

  • see PosterCircle & FingerTips for examples on Apple Developer Safari site
  • can animate multiple CSS properties — just add them to the -webkit-transition-property property
  • can also adjust timing and have a transition delay
    • delay to have things appear in a staggered fashion
  • there’s also a shorthand -webkit-transition style
  • see a good example: Falling Leaves and iPhone Safari source code

CSS animation

  • define and name animation keyframes first
  • then use them in CSS properties
  • can apply multiple animations on a single style
    • each property becomes an index-based list

Local storage in SQLite

  • check availability with window.openDatabase

Apple iPhone Tech Talk London 2009: Preparing and Delivering Video for iPhone

Eryk Vershen evershen@apple.com — Software Evangelist

This presentation went really fast with loads of detail. I captured some of the highlights below, but you’ll want to get hold of the slides to get the full detail. Supposedly Apple will be putting them up later this month…

  • challenge is to deliver better pixels, not more

there are absolute upper limits

  • 1600 Kbps
  • 48 KHz
  • stereo
  • can have alternate audio tracks (e.g. for different languages)
    • under the user’s control, not yours
  • can do subtitles — but no apple-provided tool
    • can also have alternate subtitle tracks per language
  • closed captioning not available to your code
    • anyone know the

containers

  • streaming: .ts — MPEG-2 Transport Stream
  • otherwise four other containers like .mov, .3gp

resolution & aspect ratio

  • iPhone video always takes over entire screen
  • can fit or fill video with different aspect ratio
    • fit: letterbox or pillar
    • fill: crop
  • don’t support watching movies in portrait since lose too much of screen

bit rates

  • bandwidth is not a constant
  • so would want to deliver multiple versions, using adaptive to switch during playback

preprocessing

  • there isn’t a recipe…
  • big three:
    • deinterlace
    • crop/scale
    • smoothing — simplify the image

encoding

  • make reference movies so can serve up multiple bitrates
  • quicktime 7 pro includes advanced MPEG-4 settings
    • ensure you use the baseline profile

progressive download

  • structure movie for more random access
  • map of movie moved to the beginning
  • server needs to support byte-range requests
  • works with reference movies — picks appropriate movie based on data rate
    • once the system has chosen a movie, you’re committed
    • http streaming allows you to change during a movie

streaming

  • http streaming introduced with snow leopard and iPhone OS 3.0
  • need a media encoder from someone else: Apple don’t provide one
  • segmenter:
    • converts stream into segments
    • continually updates playlist file
  • client:
    • downloads playlist
    • downloads segments
  • format:
    • MPEG-2 TS
    • also supports elementary audio
  • playlist is an extended M3U file
    • should be unicode UTF-8
    • use extension .m3u8
    • #ext-x- tags added by apple to support dynamically changing playlists
    • #extinf tag comes before each segment url and indicates its duration
  • on demand playlist just like live playlist, but has a #ent-x-endlist tag at the end
  • player currently seeks to segment boundaries

variant streaming

  • variant playlist contains references to other playlists
  • just another m3u file
  • new tag: #ext-x-stram-inf
  • includes bandwidth, program-id, codecs parameters
    • see rfc4281 for codecs options
  • client current starts with initial stream and works upwards, though this might change with future implementations
  • really useful slide

audio only

  • can include still frames in an audio only stream
  • want 3:2 aspect ratio
  • still frame gets included in each segment (so can change on a per segment basis)

encryption

  • encrypted by segmenter
  • control key access using standard HTTP techniques
  • add a #ext-x-key tag, w/ method & URI
  • will apply to all subsequent segments until another key tag appears
  • can clear with by having a tag with method:none

streaming encoders

  • supported by hardware encoders
  • don’t have a software solution that Apple are recommending at the moment
  • average segment duration: find 10s is a good number
    • can go shorter, but don’t want to go too short
    • means that you have to supply more segments, and your server will take more load
  • have to have at least 3 segments at once available in playlist
  • users can seek if you provide more

segmenter

  • Apple provide a segmenter tool
  • want to get the one from the iPhone dev center — it’s newer than built-in with snow leopard and has extra features

deployment

  • for CDNs — want to set expire time on delivery file to be about half of segment length

notifications

  • can send a highlight parameter as a notification
  • app can request that highlight to get a custom playlist
  • e.g. when a goal is scored…

playback

  • MPMoviePlayerController only does landscape
  • UIWebView can cope with portrait
  • iPod app can play portrait if you make it a video podcast
  • can’t have a local HTTP stream

html content

  • put embed tag inside video tag so old browser will cope

references

Apple iPhone Tech Talk London 2009: iPhone User Interface Design Essentials

Eric Hope ehope@apple.com — User Interface Evangelist

  • solve a problem
  • clearly defined style
    • axes: (usage) serious / fun; (content) tool / entertainment

serious tool

  • use alignment well — see address book & settings for examples
  • avoid redundancy
    • if a word is showing up more than once on a screen, get rid of it
  • prioritise info, esp. above the fold

games (fun entertainment)

  • games should be straight in — no hierarchy
    • most games are casual — you don’t know how long the user will have
  • games should be multitouch enabled — most are
    • any that aren’t are frustrating
  • lowercase “i” means go back to menu/options
  • give loud visual feedback

serious entertainment

  • stick to standard UI elements for navigation
    • the user doesn’t want to learn new things to get to their entertainment

utilities — dead centre on the grid

  • as graphically rich as possible, on a single screen
  • can they be run five feet away from you in a dock?
  • equate to single-use appliances

essential

  • one door to one room
  • want a tree, not a web
  • shared things should be in a modal sheet
  • clean layout:
    • focus on structural integrity — as if they were real physical objects
    • the human mind feels uncomfortable if the UI would fall over if it were physical

gorgeous application icon

  • one of the most underestimated and undervalued contributions
  • legible:
    • one primary silhouette — can you do it in a shadow on the wall?
    • the mind parses shapes, then colours, then words
  • high quality (fidelity) art
    • store is saturated with different icons
    • initial apps had equivalent to apple, but now full colour, hi quality
    • apple don’t need to change their apps — they’re built-in
  • good example:
    • ramp champ — didn’t sell well due to icon ambiguity
    • pretty and well-defined, but name and icon didn’t mean much
  • “your icon is your business card”
  • make sure any branding doesn’t obscure silhouette

hi fidelity UI

  • tactile design:
    • no intermediary to interface (mouse, keyboard)
    • want wood, leather, aluminium — increases perceived value of app
  • free yourself by starting with paper prototypes

dynamic content — most essential

  • your app needs a pulse
  • apps have a short shelf life on the store
    • most mac apps are serious tools
    • most iphone apps are not — they’re content consumption based
  • most users read what’s in the update before they upgrade
    • don’t just “fixed bugs”
    • even if you say “fixed memory leak that affected 1% of users”
    • it’s a “love letter to your customer”

add in-app purchases

  • adds investment to your app
  • people will come back if they’ve purchased content within your app
  • even if a user only spends 99¢ on apps a week, they could spend 10 x 99¢ within a single app — increases their return on investment
  • make sure that users can pick up their existing purchases when they get a new phone

animation

  • e.g. springboard press and hold
  • avoid continuous animation

sound

  • “the forgotten frontier for iPhone development”
  • 90% of non-game apps have no sound
  • compulsion like a mother going to a crying baby
  • e.g. mail sent sound in the background even when you’ve moved on to doing something else

polish

  • standard alert is generally a bad thing
  • icon for spotlight is 29x29 pixels — make sure you stand out
  • don’t introduce push notifications until it’s solid
    • people will disable it and never see improvements
  • support extras like copy/paste, undo/redo (generally uses shake)

app definition statement

  • (your differentiator) (your solution) for (your audience)

Wednesday 4 November 2009

O2 Litmus: Palm Pre Mobile Web Developer Event

Tonight was a really impressive event organised by O2 Litmus. The two guys from Palm did a very good job presenting WebOS and Palm’s plans for the future. The food and drink was excellent. And they even gave us a Palm Pre each to take home! Certainly makes me want to at least try out making an app.

As usual, here’s my notes for the evening in a vaguely coherent manner…

  • webkit appearing all over the place on mobile
  • as well as opera (there were a couple of people from Opera at the event)
  • HTML 5 is providing standardisation for web applications in the same way that HTML provided standardisation for web documents
  • web applications are escaping the browser:
  • why not flash, javafx or silverlight?
  • because:
  • “When you improve things by an order of magnitude, you haven't made something better — you've made something new” — Stephen Levy
  • Palm Pre uses V8 javascript engine, just like Chrome
  • WebWorkers provide background threads
    • came from Gears worker pool
    • invented to stop database access causing hangs
  • Chrome uses WebWorkers for extensions
  • Firefox hasn’t implemented SQLite, but may go for a JSON-based database, like CouchDB
  • CSS Transforms
  • “it’s not javascript people don’t like, it’s dealing with cross-browser issues”
    • anyone mention IE…?
  • “it’s not just going to be developing apps for Palm — it’s making things for the web”

some detail

  • Mojo Framework is open-source
  • Mojo uses prototype.js at the moment, but will be made nicer to use other alternatives later
  • dashboard items and popups are just DOM items
  • want to integrate apps into system — background apps

security

  • web browser provides normal web sandbox
  • applications get access to native services
  • certain APIs still need permissions granted
    • e.g. location
    • can get app to ask when API is used
  • would like to push local APIs to browser windows
  • apps are packaged and signed
    • working with developers to encrypt apps in different ways
    • would anticipate that developers would be able to opt-in to encrypt their app
    • protect against people uploading a copy of an app as their own
    • balanced against the benefit of view source on the web

app store

  • Palm would like a “web app store” to emerge
  • Palm doesn’t feel that it’s the right company to make this move
  • creating a Palm catalogue & developer program for mid-December
    • charging $50 for each app to be in the catalogue — as a spam filter
    • money goes to funding developing programme & catalogue service
    • interested in finding other “friction points”
  • can get an immediate acceptance into the web distribution of the Palm app catalogue
    • submit and get a URL straight away
    • can email/tweet other people
    • no review process
  • opening up the backend too — feeds of all the apps and charts
  • would like digg-style rating
  • a developer can choose to make an app available for specific markets
  • also aiming to provide metrics for developers, so they can see how users are choosing or not choosing their apps
  • payment:
    • right now they have PayPal
    • would like to support several options
    • want to decrease the friction

BONDI & others

  • palm works with them
  • including W3C widgets & geo
  • Palm way will be there originally, but will be switched out when
  • order depending on developer requests
  • native-accelerated CSS transforms are higher at the moment
  • “Palm pays us, but they didn’t pay us enough to sell out”

supporting open source

  • waiving cost for anyone working open source
  • $99 for developer

testing

  • O2 Litmus will be recruiting Palm Pre users for testing availability
  • DeviceAnywhere will feature Palm Pre in O2 VDL

feedback

  • devrel@palm.com
  • they already use Jira and want to open it something to the public soon
  • homebrew community will patch things before Palm do it themselves
  • there are differing viewpoints internally…

personal usage

  • like multi-tasking
  • don’t like UI latency
    • hardware is roughly equivalent to the iPhone 3GS
    • don’t have access to hardware GPU — so CSS Transforms is really important
    • will happen with a over-the-air software upgrade

multiple devices

  • Palm Pixi seems a lot nicer
  • different screen size (80 pixels shorter)
  • should design liquid layouts…
  • the future is devices in all kinds of form factors

tooling

  • there are tools for Flash — what about tools for WebGL, etc?
  • mozilla is making tools
  • e.g. Atlas from 280North
  • this week there may be something new released…
  • should flash be a native platform for apps on Palm Pre?
    • nearly supported for web pages — Adobe has shown something working already

Tuesday 3 November 2009

Mobile Device Stats

I've been looking at various mobile statistics recently and thought I'd put up a quick post on my findings so far.

I started with some work at Kizoom, analysing the traffic we see on our UK Transport services. This then led to my session at BarCampLondon7, in which I presented the session counts for mobile browsers from a sample month (September) in 2007, 2008 and 2009. I've published these results in a DabbleDB database at http://mobilesessions.dabbledb.com/ for your slicing and dicing enjoyment.

I was inspired by Bryan Rieger's article on mobiforge, where he tells developers to "Expect and manage diversity" and links to some great graphs from the deviceatlas data explorer showing how screen sizes differ across device release dates. However, fantastic though deviceatlas is, it doesn't capture the actual usage of the devices on mobile sites -- it just counts one for each device, no matter whether it's a popular Nokia or an almost unused Sagem.

I've tried to rectify this lack of popularity data by publishing the Kizoom sample data. It's not perfect and each year isn't quite the same sample (we've released and withdrawn various services across the time range), but it's a start. I've even tried putting the same data into IBM's manyeyes to get some alternate visualisations.

Some of the trends that I've been seeing coming out of this data are not surprising: the iPhone punches way above its weight in terms of mobile traffic, and screen sizes are getting bigger as the years go on. One trend that is slightly unexpected is the growing popularity of BlackBerry devices over the last few years -- at least it's surprising to me quite how popular they've become (showing third in the brand popularity for 2009 behind Nokia and SonyEricsson, and ahead of Samsung and Apple).

There's been some other data published recently that seems to support the same trends. The September AdMob metrics have recently been released and this month they're highlighting the change in devices from 2007 to 2009 (a popular theme!). Their stats show that the iPhone and iPod Touch are number one and two in the UK -- probably due to in-app ads supplementing the web usage, especially when considering that the HTC Android phones (not nearly as popular a phone as the Nokia N95) also manage slots in the top 10.

Ian Homer of Bemoko pointed out the StatCounter Global Stats site, which has some very nice stats and pretty graphs to go with them. This site provides sliceable worldwide stats from 3 million websites (240 million hits in UK). Since the sites may not be mobile specific, I would expect mobile devices used to browse "the full web" would take precedence over smaller, more phone-focussed devices.

Here's an example: Top 9 Mobile Browsers in Europe 2008/09

As expected, the iPhone and iPod Touch (or "iTouch") gets the majority of the traffic, with Nokia coming a strong second. NetFront is the browser used by SonyEricsson smartphones (and others) so comes fairly high in the rankings (it seems that StatCounter may have changed how they categorise SonyEricsson browsers in August 2009, as NetFront swaps with SonyEricsson -- this is even more apparent in their Mobile OS stats for the same region).

Once more, BlackBerry devices are making a strong showing in recent months, a gain which shows even stronger when the stats are shown just for the UK.

Monday 26 October 2009

BarCampLondon7: Alternative Input - Striking a Chord

Stephen Nicholas

3rd year project

  • Stephen works with Dale Lane @ IBM on microbroker & GUI testing
  • multitap & mini-qwerty not really suited for mobile input…
    • qwerty not really suited for desktop input either :-)
    • multitap gets up to 20 WPM
  • braille keyboard has one key for each dot
  • can get faster than qwerty keyboard input
    • qwerty 80 wpm
    • stenographers much faster: 200-300 WPM
  • can possibly reduce RSI since not moving your arm quite so much
  • drawbacks:
    • have to use non-obvious input schemes (no hunt and peck)
  • microwriter, introduced in 1978
    • got speeds up to 40 WPM after 7 months practice
    • had great mnemonic method for encoding
    • recently been redeveloped as the CyKey
  • twiddler
    • has 3 x 4 keys, might work well on mobile keyboard
  • ChordTap for phones:
    • has three additional buttons on back of phone to distinguish between multitap letters

BarCampLondon7: Social Manipulation on the web and in person

Tim Nash

Social Engineering is still just as valid as ever…

  • If asked for photo ID, people will give it up willingly
  • given a photo ID with details, you can call up somewhere and say you’ve forgotten your ID
  • often the front desk will give you a new ID…
  • people are more likely to buy when there’s an official badge…
    • doesn’t really matter what the badge is
    • virtually no-one who clicks on the badge actually purchases
    • but buy rate will increase anyway
  • paypal did an experiment about a year ago
    • to see if it made a difference to not use the word PayPal on their “paypal verified” badges
    • made no difference to sales conversions
    • click-through rate to paypal went down
  • certain colours affect buying moods
    • brain doesn’t like bright colours
    • big red “buy now” button does not attract people — your eyes will avoid it
    • instead have a big grey “buy now” button with two red arrows on either side
    • eye tracking study shows people look for price and “buy now” button
    • sales rates go up when the price is just above the buy now button
  • best place for comments on a blog:
    • new comment box just under post
    • other comments can be down the side
  • use the word “reply” rather than “submit” comment
  • scienceforseo blog
    • highlight a part of the text and the comment will appear alongside

BarCampLondon7: Project Lombok - the end of Java boilerplate?

Reinier Zwitserloot - @surial

Project Lombok is an extension to Java that allows you to write less boilerplate code by using annotations.

  • examples:
    • @Data to provide automatic getters and setters for private fields, equality and hashCode, and a constructor for final fields
    • @Cleanup to tidy up inputstreams when block ends
    • @Synchronized to lock on private Object instance rather than this
    • @SneakyThrows() hides a checked exception from javac, but leaves it for JVM
    • (in JVM all exceptions are unchecked)
  • integrates with Eclipse (e.g. getters/setters provided without you having to type)
  • does mucking around through its jar, so can work easily with Ant & Maven etc
    • use com.sun...ProcessingEnvironment to
  • Java Posse — podcast
    • inventor of annotations happy that lombok exists, but not so happy that private APIs being used…

BarCampLondon7: Supporting the masses

Tony McCrae

Experience from squadlist.co.uk — an online tool for organising rowing outings. Set up during a Guardian employee’s spare time for his own rowing club, but suddenly getting over 1000 users. He needed to support the users of the site without spending time on them.

  • sort out forgotten logins
    • though have to be careful about security
    • generate nice passwords using pwgen
    • facebook isn’t permitted in some firms (so no facebook connect)
    • prompt end users to talk to someone else other than you!
  • can your users be divided into groups?
    • form a relationship with one member in each group
  • demo as documentation — install a full demo system with sample data and let them play
    • reset its data regularly (but make sure you tell people!)
    • Jira now has jQuery-based inline popup help
  • invest in getting your domain unblocked
    • put SPF in your DNS
  • even if you send from noreply@, check the volume going to it!
    • a spike may indicate something going wrong
  • don’t tell everyone when changes happen
    • just senior users
  • introduce new features to a smaller test group
    • enable beta group
  • allow users to export their data automatically
  • http://uservoice.com and http://getsatisfaction.com
    • with a forum on-site, people keep on requesting the same new features, even if you provide a roadmap
    • moving to these customer satisfaction sites means that existing feature requests no longer generate complaints!
    • instead, people vote up the existing requests
    • of course, if the requests never get monitored or picked up, you’ve still got disgruntled users
  • really loud heavy users can be very powerful for good or bad
    • hire them! or at least make them happy
  • gmail labs has “canned responses”
  • open source core product to ensure continuity
  • provide “paid for” additional capabilities

Saturday 24 October 2009

BarCampLondon7: Release London's Data -- Feedback from #londonsdata

Emer Coleman (assistant chief exec for London Borough of Barnet)

  • has been doing research this year about what data has been collected in london boroughs
  • big crisis in public finance after next election — what will happen?
  • to have a meaningful conversation, you need to have information available…
  • lack of trust at the moment (esp. for MP’s finances)
  • GLA holds a lot of data and wants to publish
  • already publish financial data
  • hoping to put pressure on subsidiary bodies to release more
  • invited developers to ask for what they wanted
    • 65 developers gathering information
    • TfL & transport info is really where they want to get to
  • developers have gone as far as they can to push agencies
  • “we’re just gonna do it”
  • asked “do you want raw data, or with some contextual data?”
    • didn’t come to any firm view
  • should we just release with the cabinet office?
  • now an email list, and a google group set up
  • will be running a competition
  • aiming to release data store in January
  • data avilable
    • air quality maps
    • number of people in social housing
    • school catchment areas
  • message coming from developers — just put the data out there
  • Daniel Shore (@LDN): what’s the attitude of TfL to scraping the journey planner?
  • IanForrester: other authorities (e.g .Manchester) trying to do things too
  • DIY Birmingham — scraped version of local authority site, but much more accessible
  • Cloudmade: NaPTAN data being imported into OpenStreetMap
    • then allows it to be updated, corrected and extended
  • Beth Noveck (Obama’s advisor on transparency) has written a book about wiki government

BarCampLondon7: Non-relational Databases

Simon Willison

Back channel notes on etherpad

  • why?
    • scalability issues — have to do bizarre things to get to Flickr/Google size
    • some models don’t fit schemas
  • Voldemort — used by LinkedIn
    • needs at least four servers to get started!
  • CouchDB, MongoDB, etc
    • download and type make
    • MongoDB was much faster, tho’ CouchDB has improved
  • whenever you hit a tag page on on flickr, you hit a search
    • if you hit “my photos, tagged X” you hit a relational database
  • Programming the Semantic Web
    • by the guy who wrote Programming the Collective Intelligence — very good: all people who like X will like Y
  • redis
    • key-value store, network accessible
    • ridiculously fast
    • doesn’t persist to disk — every 15 seconds it dumps the entire database to disk
    • can improve reliability by replicating
    • e.g. live stats services
    • can have a key-set, with add to set, set intersection
  • Git
    • has shown that it can scale to the size of the linux kernel
    • so can scale to storing your desktop settings!
    • git is not just a RCS it’s a file system with revision control
    • there’s also git# and jGit
  • jaiku migrated to app engine
    • including all the history
    • need to think of queries at design time, otherwise you’re stuck and have to do a big MapReduce to extract data

BarCampLondon7: iPhone Stats

Chris and Simon gave a good description of what it’s like to be popular in the Apple App Store.

  • get an angle for the app
    • 0870 talked about in Guardian, twitter, TechCrunch, …
  • 0870 download stats available on simon’s website
  • marketing from Apple is not the be all and end all
    • and you have no control, so no warning when serious traffic hits…
  • top 5/top 10 is the big hit
  • when you put out an upgrade, 60-70% of new downloads will hit your app (and your feed)
    • again you have no control of when this happens
    • it’s usually at night (in the UK)
  • hosting:
    • simon started on slicehost
    • went down after a couple of hours…
    • moved to rackspace cloud (engineX)
    • their cheapest server — has cost about $10 so far
  • Kieran: tune apache so it can handle more connections at once since mobiles will talk for longer
  • Ads:
    • good at the beginning but tailed off rapidly
    • Simon looking at premium ad suppliers
    • need 500K impressions/month
    • made $800 this month, but could make more on a premium network
  • Q: Is it a point against you if you release your app on Cydia?
    • A: don’t think so, but then haven’t tried
  • stats for paid:
    • if you’re lucky you’ll get 1000 a day
    • that will get you into top 25
    • a little more will get you top 10
    • if you’re focussing on a single territory, make it the US!
  • iPhone apps are a great marketing tool
    • but probably won’t cover your costs…
    • Nigel saw a company that saw increase on their mobile web site when the iPhone app went live
  • subscriptions are rolling:
    • app will warn you that you need to purchase a renewal
  • Kieran: in-app purchasing is extremely successful — much more than subscriptions
  • Even Nike couldn’t get app fast-tracked….!
  • submitting & checking by Apple:
    • even new versions get treated as a new app
    • they only tell you one problem at a time
    • it’s about a 2-minute review — not a QA test
  • AppViz — you need it for viewing your stats
  • 0.5% of users write a review, and less than that actually write a comment
    • paid apps get more reviews — people have invested
    • people don’t know how to do reviews unless they delete the app
    • appirater is an open source library that will prompt people to put in a review and take you to the right place on the app store
    • definitely put a feedback screen in the app
  • putting apps cheaper at the beginning doesn’t seem to make much difference
    • there’s a bit chasm from free to paid, but then once you’ve made it paid there’s much less of a drop-off

BarCampLondon7: Energy Efficiency & Usage Monitoring

Nigel Crawley

  • digital meter — LED flashes fast or slow depending on how much electricity you use
  • can pick that up with an arduino with wifi and then make available as EEML
  • EEML (eeml.org) — XML for electricity cost
    • can humm output eeml?
  • can then input into Pachube
  • can then do visualization like this: BBC spiral viz of podcast #bcl7 on Twitpic
  • lilypad arduino — can sew into clothes
    • can include a vibra-ball
  • can recognise individual devices by whole home electricity usage
    • fridge, kettle, toaster, electric oven
  • Tom Raftery — greenmonk.net
    • devices using too much electricity — an organisation will offer to replace it and tell you the savings
    • Camden, New Jersey: government offers subsidies on lower energy appliances
  • visualisations:
    • one for schools that showed a polar bear running out of iceberg
    • DisplayLink have done a blog post on energy visualisation

Gbenga Kogbe

  • the UK will run out of energy by 2014… we must save
  • comparing with your neighbours
  • mancini project — plug by plug usage
  • there was some effort in the Zigbee standard — all appliances would publish their usage to a standard hub
  • energyhive provided reduced price meters
    • research that came out showed that by the end of the trial, loads of meters were in the drawer and not used
    • several 1000 homes included in trial
  • putting information online and sharing it is much more effective than a little meter in the corner
  • in some places, there are dynamic tariffs
    • would like to tell dishwasher to wash when it’s cheap
    • not in the UK…
    • energy providers buy at realtime, but sell at flat rate
  • Dale Lane: energy costs vary between 2p and £3 a unit!!
    • it’s in their interest to get us to use it at the right times
  • base electricity is provided by nuclear power
  • peak is provided by hydro
  • DynamicDemand.co.uk: figure out national supply by checking frequency
    • brownouts caused by frequency going too low
    • looking at making fridges turn themselves off when the frequency is lower
    • if all fridges did this, then peaks would be made less
    • see also caniturniton.com
  • in California they have battery farms (since the 80s)
  • solar panels are less efficient in the heat…
    • they get powered by light, not heat
  • bike generators:
    • bikes available for free — have generators
    • when they are parked, they provide their power for the local buses

Monday 12 October 2009

MomoLondon: Go your own way? A fresh look at LBS

Tonight’s event was probably the busiest I’ve ever seen the CBI Centre — the room was packed and there were loads of new faces. In terms of the content, there were good bits and not so good old bits — the “walk past a shop and get a voucher” idea reared its ugly head yet again, despite being shown several times to be of no interest either to advertisers or to the consumers.

Vodafone showed off their 360 phone and some services, but shorn of the glitzy launch, it looked a little clunky. The panellists spent a lot of time talking about foursquare and Andrew Scott of Rummble got a few words in edgeways. I’m not quite convinced that either of them have a business model. Shazam looked impressive and gave a good demo — they do have a business model, even with Spotify around. The guy from Cloudmade knows his stuff about maps (the stuff they do is fantastic), but didn’t seem to have much of a clue about mobile.

Skyhook Wireless

Kate Imbach

  • Now selling hybrid location: Wi-Fi, Cell ID & GPS
  • 10-20m accuracy within 1s
  • Drive to find networks
  • London coverage follows people, but empty in parks
  • 125m+ APs worldwide
  • 70% population coverage in Europe
  • Commercial apps require an ad revenue share or a licensing fee
  • Free apps can use SDK for free

Some examples (all on the iPhone):

  • Sit or squat — rate toilets…
  • MyStarbucks
  • Flixster — cinema reviews w/social features + find cinemas & restaurants close by
  • Shazam tying music to locations
  • Sportacular — voting by location, then see stats over country
  • TuneWiki — see where people are listening to what music, and what’s popular around you

Advertising

  • MyTraffic — driving alerts, works in Boston
    • has smarter location-based advertising — but who is selling them?
  • Scoreloop — location-based leaderboards
  • Amazon — what’s popular in Seattle this week? what are the business books selling in London?

http://www.skyhookwireless.com/developers/developersguide.pdf

New launch — Maps Booster for S60

  • feeds Skyhook location into all location APIs on the phone

Vodafone 360 Demo

Patrick Weissert

  • WiFinder acquired by Voda and providing location for 360 phone
  • VF 360 widgets (see http://jil.org) have location API available
  • Apparently Vodafone 360 is still beta…

Flook

Tristan Brotherton

  • browse photos (“cards”) around you and see where they are
  • can add comments to the cards
  • have a follow model like twitter
  • can collect cards you find interesting (favourite)
    • can then add categories
  • scoring mechanism
    • if other people collect your cards, you score more
  • seems too much like twitter with twitpic
  • (integrates with twitter)
  • what’s the business model?

Shazam

Jonathan Symons

  • > 35m users
  • live in 150 countries
  • Jonathan’s role is to think of the money…
  • money comes from:
    • users getting involved, sharing app & music
    • buying things
  • getting roughly 60,000 tweets of music a day
  • “I can find myself in a bar”…
  • meet new people who like the same music
  • basic service for free, funded by advertising & follow-on content acquisition
  • additional services cost

Panel Session

Panellists:

  • Stuart Dredge of Mobile Entertainment
  • Kate Imbach, Skyhook Wireless
  • Jennifer Hicks, Ink Communication
  • Nic Black, Cloudmade
  • Roberta Lucca, Vertu
  • Patrick Weissert, Vodafone

Vertu

  • just launched a new phone
  • together with a location awareness service
  • Vertu consumers work hard, play hard, spend hard
  • travel way more than average
  • not digital natives — use things that are relevant for them
  • want recommendations & whole experience, completed with human concierge

Cloudmade

  • here to lower the barriers to entry for location apps
  • making location a feature, not a whole application

Ink Comms

  • Global Marketing & Strategy
  • analysis

Is iPhone leading the way? What about Android & Symbian?

  • KI: a lot of focus on iPhone
    • Android devs scaling back and going for iPhone instead, since that’s where the money is now
    • Lots of faith in Android but it will take its time
  • NB: various limits for iPhone, esp. everything has to be active
    • Android allows swapping out services for others, e.g. using Cloudmade map service instead of Google
  • PW: VF360 addresses 3 out of 6 segments
    • iPhone is making mainstream what we thought a couple of years ago was just for geeks
    • Android will take similar path
    • can share VF360 even if you’re not on Vodafone (and it picks up facebook, etc)

Foursquare

  • JH: foursquare may come up with a biz model with advertising on a local level
  • NB: foursquare about building up local content
    • need it for developers to play with
  • RL: content useful and so is location, but targeting is vital — need to match the right content according to the influencers
  • NB: have a choice of “what don’t I show on the map”
    • depending on the activity, the time, the context
  • SD: segmenting by the apps you select
  • RL: sometimes people don’t want discover things — they want the choice made for them

Has the industry come round to starting with the service rather the tech?

  • KI: lots of complaints that the Apple App Store has too many apps, but there are a lot of niches
  • KI: 100s of millions of lookups every day — location is becoming ubiquitous within apps, becoming mainstream

How are people making money?

  • KI: Visa are sponsoring a shopping app, seeing $15-24 CPMs — does anyone know what this is?
  • JH: still trying to come together
    • lives in Amsterdam, people are using foursquare like crazy
    • social networking side of location services
  • NB: releasing first part of location based advertising service later this year
    • tailor advertising to app’s context
  • RL: need to involve audience
  • PW: flixter are ticketmaster — they sell tickets
    • the app is advertising for their own service

@torgo: What about the privacy issues? Could that lead to a backlash against location services?

  • KI: app developers are sensitive to the privacy issues
    • generally sharing location for the user experience
    • weird when app asks for location for no reason
    • Skyhook don’t do user application tracking
  • JH: consumer issues will hit as this stuff spreads
    • consumers can get picky
    • getting libel suits as twitter becomes mass market
  • NB: thinks the early adopter community are more concerned by privacy issues

How does immediacy affect social mores?

  • RL: different countries reacting differently
    • next generation will not bother about personal info
    • e.g. Google StreetMap just useful
  • KI: gave up on privacy after my mother guilted me into accepting her friend request on facebook
  • Floor: In Amsterdam, youth feel privacy is very important
  • JH: want you to ask permission to share their info
    • teens will not part with their phones
    • will not be sliced by age; depends on industry
  • PW: need to enable location selectively
    • not enable every tweet
    • choose to

How does Vertu work with location?

  • RL: Vertu handsets are Series 40, but luxury
    • use of location is made clear — transparency
  • SD: more like “push my location to the paparazzi right now!”

What is the thing that O2 UK could do that would be most helpful?

  • NB: reduce barriers to entry: make location part of the platform, and pay them for using it, via location

Does advertising really work?

  • A user is in Kensington, shoe shopping and I want to serve them an ad. Right now, advertisers might have an advert about shoes, probably clothing instead, almost certainly not Kensington
  • NB: might also want to tell people about coffee shops nearby
  • JH: talks about walk-by advertising…
  • @fj says “I’m not responsible for any store making money - get outta my phone!”
  • KI: no inventory
    • no incentive to sell more targeted ads as they sell less impressions that way
    • location-based banner ads are probably not the way things will go
  • PW: time & location push model doesn’t really work
    • more about pull — enabling people to find you

Will there be more integration between mobile and web apps?

  • PW: VF360 all about that. Can sync them up
  • RL: all about lead time: takes a long time to put a new device on the market, but not to launch a service

Are we going to see content fragmentation?

  • PW: biggest problem of app store model — think that services will converge
  • SD: federating content like wikipedia
  • NB: will be launching a “publish content through a central service” later this year

For certain apps, a facebook app is worth about 400 leads

  • JH: it’s too early
    • foursquare think they will make money from local advertising
    • they’re experimenting at the moment
  • Andrew Scott: foursquare not making money — just taken $100m of funding…
  • RL: concentrate on providing something that is highly valuable for their consumers

Upcoming events

  • 26 Oct — Mobile Phones for the Senior Market
    • Even in the UK there is only about 50% penetration in the 55+ market
    • Learn about the opportunities and how to take advantage of them
  • 26 Oct PM — DCKTN Mobile World Congress competition judging
  • 9 Nov — Mobile Monday joint birthday part w/Swedish Beers
  • 7 Dec — Heroes of the Mobile Screen
  • 4 Dec — OpenMIC in Guildford on AR & Location

Saturday 3 October 2009

OverTheAir '09: Fluid Layouting Techniques for Widget Development

Daniel Herzog

Original slides:

My notes…

  • There’s one web — “You don’t want .mobi — it wasn’t a big success”
  • widgets should be cross-platform
    • should take a small amount of space on the desktop, but be bigger (full screen) on mobile
  • Vodafone widget engine is based on Opera 9

Fonts

  • use media queries to switch to high dpi mode, e.g.
      @media all and (-o-touch) {
          a {padding: 1em}
      }
    • uses opera-specific -o-touch
  • alternative:
      @media all and (min-resolution: 200dpi) {
          body {font-size: 22px}
      }
      .myDiv {margin: 1em;}

Images

  • should use them at native resolution, loaded from server…
  • if you’re desperate, use image scaling using screen width percentages

UI-Elements

  • use dynamic images, or scalable SVG
  • SVG works across most browsers, apart from IE (but may change in 2009)
  • Google working on SVG Web — uses Flash plugin to render SVG content
    • But if widget engine doesn’t support SVG, it may not support Flash either
  • Daniel uses Illustrator to create initial SVG, then uses TextMate to hand edit it…
    • Can also use Inkscape but the UI is horrible

More resolutions…

  • Portrait & landscape, docked mode (also in portrait and landscape!)
  • Deal with these using window.resizeTo
  • Call this initially and when the orientation changes

Future

  • We’re in trouble — widget runtimes need to return real dpi/ppi values
  • Not possible at the moment since developers assume that 10pt is a certain number of pixels
  • Would be nice to have an extra CSS property for the real dpi

And another thing… zooming widgets

  • Zooming enlarges pixels — see Opera for an example
  • But SVG still scales effectively
  • Another example: WidgetCity is a widget version of Sim City!

OverTheAir '09: Natural Interaction Gestures

Joe Macleod & Younghee Jung — Nokia Design Team, London

A though-provoking discussion on how interaction with handheld technology develops. Nokia seem to be doing some good research on user interfaces — shame that doesn’t seem to be transferring into the Series60 devices…

Exploring ‘natural’ gesture interactions

  • Not really anything that replaces face to face communication yet
  • Some gestures transfer across cultures well, e.g. fingers on lips: sshhh
  • Some are complicated, e.g. transferring business cards
  • v-sign doesn’t necessarily mean victory now
    • in Asia it is now a default pose for taking photos
    • watch out for reversing it…
      Not quite the default pose for taking photos
  • in China, can count from 1 to 10 on one hand…
    Counting from 1 to 10 in China
  • in Ghana there’s a really sophisticated hand greeting ritual that includes clicking fingers together with the other person
  • research exercise in Nokia to pull together existing gestures used in phones
    • e.g. turn a phone over to silence a ring (Nokia 8800)

Initial research

  • what are natural gestures involving mobile phones?
  • selected 11 different tasks, from checking time to sending something
  • gave people blocks to play with (no buttons)
  • sketched out what people came up with in four cities around the world
  • e.g. silence gestures:
    • stare at it
    • put finger over the speaker
  • had to fight with people to get them to think outside of pressing buttons

Understanding the factors that make gestures work

  • 60 gestures as stimulus mapped to 15 tasks (4 each)
    • some were already known to be not so useful
      Sample gestures used in Nokia gesture research
  • 14 nationalities among participants
  • video recorded each gesture to control how they were presented
  • some issues were physical capability — can I do this? is it comfortable?
    • “I don’t want to feel tired using my phone”
    • “I don’t want to hit someone in a crowded place”
    • “Might I damage my phone?”
  • have loads of examples of bad gestures, e.g. skipping!

Testing with prototypes

  • Nokia Labs built a sensor packed platform to test prototypes

design principles

  • easy to learn
  • easy to perform
    • you will learn it from other people in the real world
  • no unintentional triggers
  • designed for the mobile context
  • scalable visibility
    • some people are more flamboyant than others…
    • “draw a circle” — comes in all sorts of shapes and sizes!
  • fits the task
    • even if the gesture is not from the natural metaphor, it should have some link to the task
  • works together as a language
    • people already mentioning that if there are more than 3 or 4 gestures, they wouldn’t be able to remember… unless they were natural…

Q&A

  • Isn’t it easier to learn new gestures with a new object?
    • e.g. introducing NFC cards
    • making existing actions a bit smoother
    • introducing new features
  • Do you think you could take inspiration from sign language gestures?
    • Have you ever seen deaf people talking over a video phone…? The language meant to be seen from a distance…
  • Is there anything that allows users to start experimenting with their own gestures? Like users added hashtags and other meta-data to twitter?
    • Some Japanese phones already let you map 4 pre-defined gestures to specific functions
  • Will some gestures inspire a new form factor?

Tuesday 29 September 2009

OverTheAir '09: Mobile Visual Interactivity

Geoff Ballinger — mobile acuity

Geoff runs a mobile development agency in Scotland, specialising in applying vision technologies.

  • lots of real people use the camera on their phone
  • in 16-34 age group 78% have sent an MMS, 48% cameras regularly
  • there are graveyards in Japan with QR codes on each headstone!
    • scan them to get some details about the person
  • examples:
    • take a picture of a goal and MMS it — it responds with a mobile web page that tells you whether you scored, then pulls you in to further interaction
    • Nike PhotoiD — find some colours you like, send it in and get a picture of trainers coloured that way
  • Visual search apps on iPhone store (CDs and books):
  • recognise images from database of 10m
  • code libraries
    • Haar recogniser
    • ARToolkit
  • web service APIs: Kooaba & Mobile Acuity (available on a per-use basis)
    • feature recognition from images

OverTheAir '09: Modelling the User Experience

Bryan Rieger — yiibu

Bryan comes from a theatre and animation background and was sorely disappointed by the tools available for designing interactive services. He and his partner Stephanie have developed a lightweight prototyping mechanism for interactive mobile services. I like the idea — especially the bit about throwing away lots of ideas in order to explore.

Tools for traditional animation

  • scripts & storyboards —> models & layouts
  • adding structure, but always changing
  • dope sheets, now known as timelines
  • pencil tests
  • animatics — complete but not finished
    • stills set to music/voice over, possibly panned and scanned

Tools for Mobile design

  • wireframes and… more wireframes
  • what are they?
    • layout, behaviour and flow all in one document
  • are we expecting too much from one document?
  • Bryan started dropping little nonsense text on the descriptions of wireframes to see if anyone was reading them — turns out nobody was!
  • documentation gets bigger and bigger but not readable
  • document management != design

Prototypes

  • where do you start and where do you stop
  • paper -> HTML -> Flash -> Python, Java, …
    • the more you get to development, the more people don’t want to tear it down
    • developers (and those who manage them) should learn the value of the spike — a throwaway prototype that you really do throw away
  • want to get something more concrete than paper but has limitations
    • really difficult to share
    • usually requires some mediation

Agile

  • iterations are good for design
  • you can explore more, develop new different ideas and throw away bad ones
  • how can I make lots of mistakes? how can I get visibility of the project as a whole?

Integrating knowledge from animation

  • applying animatics to wireframes
    • want something interactive, but still sketched
    • and the sketched part must include the flow and behaviour as well as the layout
  • disposable data models:
    • views, states and events
    • views and states are scanned in from paper prototypes
    • events switch from state to state or to different views
    • creating these is relatively easy, esp. using Fireworks as a tool
    • Fireworks allows you to create multiple states side by side, all based on a single view
  • can switch in different JPEGs really easily — iterate visual design
  • can easily alter model (flow & behaviour) by changing XML
    • at the moment, the XML is hand-built
  • player built using Mobile Processing, so runs on real devices (as JavaME midlet)
    • same player for any design — just load and go
  • models (including graphics) take about 1 day to create
    • so can create 60+ models over several weeks
  • is this system similar to Cascade Breeze, or PhoneGap?

Experience

  • created lots of documentation up front, but then started culling pages
  • shipped prototypes with the documentation
  • created 60+ models — allowed change
  • took 2 designers 9 weeks of work
  • led to clearer wireframes
    • but didn’t replace them — this is an additional tool
  • much greater user testing since lots of people could use it
  • slowly refining the tool:
    • adding gestures and touch interactivity
  • don’t model the whole app — just specific flows

Wednesday 26 August 2009

MomoLondon: Mobile Java

I was delighted to be invited to be part of the panel this evening — and at only a few hours notice too (I only checked my email in the afternoon after getting back from holiday the previous evening). A flurry of tweets confirmed my place — thanks to Alex Craxton for thinking of me!

Consequently, the notes for this evening are briefer than normal, though the evening felt like it went fairly well. The topic was more technical than most Mobile Mondays and the suit percentage in the audience was very low. I hope we pitched the discussion at the right level. Thanks also to my fellow panellists: Simon Nicholson from Sun, Sergio Falleti from Future Platforms, Kieran Gutteridge from intohand and Andy Milloy from SurfKitchen.

There were a few new points raised (I’ll cover some below), but the most controversy in the room came when we got around to discussing app stores and the iPhone. There was the usual discussion around why the JavaME industry hadn’t advertised downloading apps until Apple showed the way — as Dan Appelquist put it previously, it would have been a documentary rather than a 30 second advert…

@fj questioned what Sun would be doing to catch up with Apple’s speed of development on the iPhone, but there isn’t really an answer — JavaME improvements come slowly as they involve multiple (often competing) companies working together through the often cumbersome JCP process. Apple can make changes daily. However, the JCP process eventually results in a large number of companies putting out lots of devices into multiple markets. This will inevitably cover a larger market than Apple ever wants to reach — @edent’s oft repeated point that there are many more low-level phones worldwide than there are smartphones will still be true for quite a few years yet.

As an audience member pointed out, not many of these low-level phones are used to download apps — it’s a big market but with no way of reaching it. I think this will gradually change, and we’ll start to see even the low-level phones having a reasonable JavaME capability over the next few years (this is definitely in Sony Ericsson’s plans). We’ll have to wait quite a while, though, as phones last a long time in some markets…

In other news…

Surfkitchen have recently developed a home screen app for Sony Ericsson phones on Telstra in Australia (called Telstra One). This plugged into a new API only available on those phones. I wonder how the new app affects battery life — and if it copes well, when this new API will be released. Sony Ericsson seem to be moving forward in JavaME in the way that the JSR community process cannot, but still creating an effective platform based on the JSR standards. I like their approach and I hope they do well.

Device databases

Sergio from Future Platforms said that they use the Java Verified list of device families and lead devices as a basis to categorise their builds. I wonder how often this list gets updated and how FP deal with newly released phones. This device grouping is something that should be part of a device database — something that’s sorely needed to enable developers to choose which builds to make in order to cover an appropriate range of devices. If you choose effectively you can massively reduce the testing required. Most development agencies have built their own databases, but aren’t that willing to share it as it’s part of their competitive advantage. The JavaME 3.0 SDK has a database included, but it’s not detailed enough and doesn’t get updated very often. J2MEPolish have an open source device database, but again it’s not updated that frequently and it also contains mistakes. WURFL and dotMobi have more up-to-date device databases, but they’re focussed on mobile web and don’t have the information required by Java developers. And developers need to be able to slice and dice the data to pick out appropriate pieces. Anyone for putting it all into a DabbleDB database?

New attack on fragmentation

One of the reasons that we need a comprehensive device database is that manufacturers implement their mobile Java VMs in slightly different ways. The existing official tests that mean they can put the Java name on their product seem pretty low-level and certainly don’t cover a lot of the user interface behaviour. Several manufacturers also seem to have a lower level of quality control for their Java implementations and bugs get through to devices with alarming regularity.

Orange, Sony Ericsson, Sun and Vodafone have got together to create a new set of tests, this time created by ordinary developers rather than JCP members. The aim of JATAF (you really don’t want to know what that acronym stands for) is to collect anti-fragmentation tests and release them as suite that can be run by manufacturers, operators and developers. I’m not sure whether anybody will be enforcing devices to pass the JATAF suite, but the participation of Orange and Vodafone is a good start.

Upcoming events

  • Ecomo September 11th & 12th — there will be netbooks as prizes
  • OverTheAir September 25th & 26th — still looking for more sponsors