Apple iPhone Tech Talk London 2009: Overview

Apparently I can blog about this year’s Apple iPhone Tech Talk World Tour, so here’s my notes.

This was a much larger event than last year — and I wasn’t the only person surprised by how many developers turned up. Since the registration process involved entering an iTunes URL, most of the people present must have written an app and got it into the store. However, the gender balance was even worse some recent developer events I’ve attended — it looked like there were more women on the Apple team than in the whole audience…

The kickoff session was a quick rundown of where iPhone development has got to since it launched only 15 months ago (seems like longer, possibly because the jailbreakers got there a fair amount earlier…), and the rest of the day was spent going into details (I'll cover the sessions I attended in later posts). I really hope the Apple guys put their presentations up soon, as there was a lot of great detail that I couldn’t type fast enough to get down :-)

stats

• only 15 months since App Store launched (though an extra year of iPhone)
  • 2 Billion apps, means 180,000 apps an hour…
  • 100,000 apps in app store
• more than 50 million devices
• 81 countries
• 125,000 registered developers
  • so at least 25,000 who paid money but haven’t put anything in the store yet!

timeline

• jun 08: seeding SDK & emulator to all devs
• jul 08: app store live
• sep 08: customer reviews only by purchasers & versions
• oct 08: crash reporter & logs
• mar 09: developer discussion forum (still beta), more customer service agents
• jun 09: quick reference guides
• jul 09: news & announcements + better developer support
• sep 09: new app store categories (genius, essentials), top grossing lists, app store resource centre
• oct 09: in-app purchase available to free apps

the best apps are…

• simple or sophisticated (hiding complexity)
• lightweight
• intuitive
• small pieces loosely joined
  • lots of separate features
  • what about feature interaction? is that going to be a problem?

iPhone users have changed

• not necessarily technically savvy
• bringing technology into new places

specific examples of innovative apps

• each app has 5-7 seconds to sell itself to the user
  • it helps to have a memorable aspect
• use the latest technologies
• go the extra mile

• keep it fresh

• urbanspoon

  • shake to randomly choose a restaurant in the style of a fruit machine
• peaks
  • Augmented Reality (AR) for mountains
• car finder
  • AR for finding car
• zipcar
  • book and find cars
  • unlock the car over network, when you’re close to it
• snaptell
  • picture identification
• postage ~ postcards
  • take a picture and send an electronic postcard
  • can apply simple visual effects
  • visually interesting error messages (look like envelopes with postmarks)
• timetuner
  • bedside radio
  • location aware — will take timezone into account
  • can set up custom alerts — different radio stations on different days
  • simple sleep timer on main screen
• weightbot
  • way over the top UI polish…
  • real sounds
  • makes it fun to enter your weight
• tweetie 2
  • only presents UI when you need it
  • swipe down to refresh at top
  • swipe across a tweet to see options
• guitartoolkit
  • swipe across to play strings for a selected chord
  • has tuner
  • has metronome that keeps playing when screen goes dark
  • can do this by setting app type — determines what happens when
• 1112
  • exploration game with lots of extra touches
• ocarina
  • with worldwide
  • and sonic lighter — added messages for choice of flame to mean something
• roambi
  • data interaction with data from SAP, Salesforce etc.
  • lots of visual interaction with data
  • pull through folders, seeing preview of front one
  • pie chart can swivel round to see details of the one on the bottom
  • charts can
• things
  • auto synchronized when iPhone comes into same WiFi network
• MLB.com At Bat
• Flick Fishing
  • using GameKit to hook up to other iPhones
• myStarbucks
  • can send your standard order to someone else so they can order for you
  • (not currently available in UK store)
• ramp champ
  • in-app purchases
  • shows preview, details and price
  • very good catalogue implementation
• iceberg reader
  • iTunes puts up a confirmation dialog of purchase as well
  • shows progress of download in-app
  • (again, not yet available in the UK store — though individual books are)

Apple iPhone Tech Talk London 2009: Networking: From Sockets to GameKit

Paul Danbold danbold@apple.com — Evangelist

This was another really fast presentation with loads of detail. I look forward to grabbing the presentation from the iPhone dev site.

• don’t want to use CFSocket or BSD Socket
  • don’t let you bring up wireless, or involve security
• service discovery:
  • NSNetServices, CFNetServices, dns-sd, BSD Sockets
  • dns-sd is Bonjour or Zeroconf

URL Loading system

• NSURLRequest, NSURLConnection & about 10 other classes (also mutable ones)
• there’s sample code for validating URLs
• recommend to use asynchronous API style:
  • use delegate mechanism — calls connection:didReceiveResponse:
• also connection:didReceiveAuthenticationChallenge:
• finally connectionDidFinishLoading:
• default behaviours should work fine

Local area networking — service discovery

• kick off NSNetServiceBrowser in asynchronous mode, w/delegation
• browser searchForServicesOfType:inDomain:
  • e.g. type _http._tcp in domain local. — don’t go beyond any routers
  • can look for your own types too such as _foo._tcp
• then get myServiceBrowser:didFindService:moreComing:
• call NSNetworkService getInputStream:outputStream to resolve
  • this takes time (many seconds)
  • wait for the user to choose before resolving
  • and let the user decide when to cancel
  • resolveWithTimeout:0.0
• must release input & output stream objects returned by NSNetService
  • bug in Apple’s code — doesn’t obey normal Cocoa behaviour

OutputStream & InputStream

• NSStreamEventHasSpaceAvailable is probably the interesting one
• though you want to handle the errors too…

advertising & publishing

• advertise service with NSNetService initWithDomain:type:name:port
  • domain can be empty — means local
  • name can be blank too — will use device’s iTunes name
• then call publish on service

stop when you’re done

• browsing is fairly lightweight on the network, but you should stop anyway
• the same goes for publishing, when you’ve got all your connected clients

register types & ports

• to register a service type:
  • http://www.dns-sd.org/ServiceTypes.html
  • send them an email, no money changes hands
  • and http://www.iana.org/assignments/port-numbers for one further

game kit — peer to peer

• 3.1 adds WiFi support as well as Bluetooth
• GKSession initWithSessionID:displayName:sessionMode
  • session ID is your Bonjour service type — how you advertise yourself
  • again, can leave displayName as nil for iTunes name
  • sessionMode can be peer to peer (1 to 1) or client/server (multiple clients)
• can’t physically go over 10-20 clients with Bluetooth
  • 3.1 and later is a lot better, but still no more than 3-4 clients
  • Bluetooth also has low bandwidth
• set up delegate and set available to YES
• check other peers with peersWithConnectionState:
  • can be available, already connected, in the process of getting connected
• connectToPeer:withTimeout:
  • generally set timeout to zero to let user cancel if they want
• actual connection made with session:didReceiveConnectionRequestFromPeer: and acceptConnectionFromPeer:error:
• must monitor the network using session:peer:didChangeState:
  • other players can drop out, so you’d have to tear down your session
• send data with sendDataToAllPeers:withDataMode:error: and sendDataToPeers:...
  • can choose unreliable (UDP) or reliable (TCP) data modes
  • as always TCP imposes a performance cost so use sparingly
• receive data with setDataReceiveHandler:withContext: and receiveData:fromPeer:inSession:context:
• always check for errors — buffers fill up on low bandwidth networks
  • check it by using a slow network
• when you’re finished, tear down with disconnectFromAllPeers:
• there’s a GKPeerPickerController for setting up Bluetooth peers
  • doesn’t work for client/server or WiFi
• the only way to find out if Bluetooth is turned on is to use the GKPeerPickerController (it uses private APIs…)
• all data sent & received is NSData-wrapped — design your packaging for network efficiency first and coding efficiency second

Voice chat

• GKVoiceChatClient and ~Service
• sets up socket interface
• handles microphone, echo suppression, etc

network challenges

scoped routing and reachability

• iphone 3.0 and later provides scoped routing
  • keeps both cellular and wifi up at the same time — so cellular connections keep going when you enter a wifi hotspot
  • but this puts a toll on the battery, so you should behave and check for networking changes
• use SystemConfiguration APIs to monitor network state
  • Am I on the network and what kind of network is it?
  • cannot tell you that packets will arrive…
  • if network changes, finish what you’re doing, close connection and start a new one
• use SCNetworkReachabilityRef and check for flags
  • but again use callback APIs since the calls take about 30 seconds
• no point using this to pre-flight as things may change
  • just use to monitor
• can set UIRequiresPersistentWiFi key
  • keeps WiFi live while your app is live
  • shows up UI elements automatically (like offering to turn off Airplane mode)
  • but if the device sleeps, you still lose your WiFi connection

battery use

• listening is cheap, transmitting is expensive
• 3GS sending leaves the antenna in high powered state for 5 seconds
  • so don’t send every 5 seconds — it will leave it in high power continuously
• compress data, send large chunks
• tear down connections when reachability changes

other tips

• try to use the cache as much as possible
• try to resume downloads rather than restart
• use pipelining if poss.
  • ask for lots of things at once and be careful about return order
  • some web proxies and apache configs just break pipelining
• also use multiple connections
• but different network conditions and different servers require different tuning
• test in areas where you know the network is flaky!
• use a packet analyzer — see QA1176
• try to isolate the user from network problems
  • often there’s a transient glitch — try again before alerting the user
• see also Technote TN2152
• look for developer forum postings by eskimo1 :-)

there’s some new sample code available:

• SimpleNetworkStreams
• SimpleURLConnection
• SimpleFTPSample
• Reachability — there’s a new version! The old version had bad code :-)
• BonjourWeb
• WiTap — peer to peer, but doesn’t use GameKit
• GKTank
• GKRocket — work in progress, with voice chat too

Apple iPhone Tech Talk London 2009: Testing and Debugging Your iPhone Application

Just before this session, I had an interesting discussion with Mike Llewellyn, author of Broadersheet, about unit testing. He uses defensive code with asserts and logging and doesn't think there is enough value in unit testing for mostly UI code.

However, Apple have converted to using unit tests and are making it easy for XCode developers to take advantage of them.

At Kizoom, we’ve used unit tests to test our controller code — ensuring that given a particular input, it goes to the right result view with the appropriate model. This is made easier by using three20 — the result view is identified by a string rather than an object (though using OCMock would let you assert about objects too).

Here’s what Michael Jurewitz jurewitz@apple.com — App Frameworks & Dev Tools Evangelist had to say:

• thousands of tests in Core Data
  • roughly 90% coverage (that’s very impressive — it’s hard to maintain more than 80%)
  • fully test driven from the start
• similarly large amount of tests in WebKit
• Application tests get injected into application and can drive running app

static analysis

• has found and fixed thousands of bugs in Snow Leopard & iPhone OS :-)
• use it to learn and enforce good Cocoa practices
• detects dead stores: unused variables
• see “Run Static Analyzer” in Project properties / Build
  • might not want to have this for all configurations
  • gets picked up by xcodebuild on the command line

organizer

• check the console if you get a crash
• can download app’s data from actual device:
  • check Applications under Summary — your app will have a disclosure arrow

beta testing

• create a distribution certificate in the iPhone portal
• create an adhoc provisioning profile
  • can still use a wildcard identifier
• load it up into XCode
• make a beta testing config
  • duplicate the release config
  • set Code Signing Entitlements to Entitlements.plist (a new file)
  • use appropriate provisioning profile
  • create new Entitlements: set get-task-allow to false
  • disallows debugging
  • if you want people to debug, then send them a debug build…
• build + send zip of build + provisioning profile to beta testers
• they can unzip & drag both to iTunes

crashes

• can get crash log from user — iTunes syncs them
  • Mac & Windows store them in different places, but can find…
  • also available in iTunes Connect
• can get Organizer to symbolicate crash log for you
  • your .app and .dSym must be locatable by Spotlight — so keep them

low memory logs

• see count of pages: each page is 4Kb
• SpringBoard shares memory with your app — it handles presentation on screen
• mediaserverd is iPod
• if you find that something other than your app or SpringBoard is the largest memory user, then file a bug…

other crashes

EXC_CRASH (SIGABRT)

• 0xdeadfa11 — termination due to too much memory
• 0x8badf00d — watchdog timeout

EXC_BAD_ACCESS (SIGBUS)

• objc_msgSend — happens when you message a deallocated object

instruments

• Zombies template — record reference counts & enabled NSZombie detection
  • see extended detail view using rectangle icon at bottom of window
  • shows stack trace for each event
  • can jump through to code view, then jump into XCode editor

Apple iPhone Tech Talk London 2009: Integrating Web Content into iPhone Apps

Vicki Murley — Safari Technologies Evangelist vicki@apple.com

• iTunes Music Store home page & GarageBand lessons are both HTML-based — included into mac apps
• UIWebKit is not Safari — doesn’t support new windows & tabs + different memory management

Loading local resources

• webview load: baseUrl should be initial path so that relative URLs work
• can’t refer to local resources in remote content
  • but can have overrides in local resources

Performance optimisation

• minimise the total number of UIWebViews
  • use AJAX
  • build views offscreen then animate them in, and delete unused
  • see examples in DashCode
• no explicit cache control, so can’t free up memory easily
  • but can set javascript objects to null

Authentication

• can embed in URL, e.g. http://user:password@www.example.com
• to prompt the user, use NSUrl APIs
  • get didReceiveAuthenticationChallenge
  • can persist credentials for session

Javascript to Objective-C and back

• evaluating javascript in Objective-C:
  • try to keep javascript simple — just a function call to a separate file
  • make sure .js files are copied, not compiled
• can go the other way round too:
  • use Javascript to change to a custom URL (still http:// though)
  • use webview delegate: shouldStartLoadWithRequest
  • return NO to say stop processing

using images more effectively

• be aware that file size is not memory size — especially for JPEGs
  • in memory, images are stored as pixel by pixel values — no compression at all
• use generated images through CSS
  • CSS gradients — they support transparency too
  • CSS masks — properties analogous to background
  • can use gradients or SVG images for masks
  • CSS reflections: -webkit-box-reflect
  • again, use gradient as final parameter
• iPhone uses Helvetica Neue for standard interface
• make buttons using CSS:
  • round corners: 10px is about right
  • add slight shadow behind text: text-shadown: 0px 0px 8px black;
  • border at bottom: -webkit-box-shadow: 0px 2px gray;
  • white overlaying gradient
  • disable standard text behaviour:
    • -webkit-tap-highlight-color: transparent
    • -webkit-touch-callout: none
    • -webkit-user-select: none
  • also make it behave like a button when you press it
  • have an enabled and disabled backgroundColor using onTouch

• see DashCode for other examples

• even if you have larger resources in total, having fewer separate resources can make the page load faster

CSS transitions

• see PosterCircle & FingerTips for examples on Apple Developer Safari site
• can animate multiple CSS properties — just add them to the -webkit-transition-property property
• can also adjust timing and have a transition delay
  • delay to have things appear in a staggered fashion
• there’s also a shorthand -webkit-transition style
• see a good example: Falling Leaves and iPhone Safari source code

CSS animation

• define and name animation keyframes first
• then use them in CSS properties
• can apply multiple animations on a single style
  • each property becomes an index-based list

Local storage in SQLite

• check availability with window.openDatabase

Apple iPhone Tech Talk London 2009: Preparing and Delivering Video for iPhone

Eryk Vershen evershen@apple.com — Software Evangelist

This presentation went really fast with loads of detail. I captured some of the highlights below, but you’ll want to get hold of the slides to get the full detail. Supposedly Apple will be putting them up later this month…

• challenge is to deliver better pixels, not more

there are absolute upper limits

• 1600 Kbps
• 48 KHz
• stereo
• can have alternate audio tracks (e.g. for different languages)
  • under the user’s control, not yours
• can do subtitles — but no apple-provided tool
  • can also have alternate subtitle tracks per language
• closed captioning not available to your code
  • anyone know the

containers

• streaming: .ts — MPEG-2 Transport Stream
• otherwise four other containers like .mov, .3gp

resolution & aspect ratio

• iPhone video always takes over entire screen
• can fit or fill video with different aspect ratio
  • fit: letterbox or pillar
  • fill: crop
• don’t support watching movies in portrait since lose too much of screen

bit rates

• bandwidth is not a constant
• so would want to deliver multiple versions, using adaptive to switch during playback

preprocessing

• there isn’t a recipe…
• big three:
  • deinterlace
  • crop/scale
  • smoothing — simplify the image

encoding

• make reference movies so can serve up multiple bitrates
• quicktime 7 pro includes advanced MPEG-4 settings
  • ensure you use the baseline profile

progressive download

• structure movie for more random access
• map of movie moved to the beginning
• server needs to support byte-range requests
• works with reference movies — picks appropriate movie based on data rate
  • once the system has chosen a movie, you’re committed
  • http streaming allows you to change during a movie

streaming

• http streaming introduced with snow leopard and iPhone OS 3.0
• need a media encoder from someone else: Apple don’t provide one
• segmenter:
  • converts stream into segments
  • continually updates playlist file
• client:
  • downloads playlist
  • downloads segments
• format:
  • MPEG-2 TS
  • also supports elementary audio
• playlist is an extended M3U file
  • should be unicode UTF-8
  • use extension .m3u8
  • #ext-x- tags added by apple to support dynamically changing playlists
  • #extinf tag comes before each segment url and indicates its duration
• on demand playlist just like live playlist, but has a #ent-x-endlist tag at the end
• player currently seeks to segment boundaries

variant streaming

• variant playlist contains references to other playlists
• just another m3u file
• new tag: #ext-x-stram-inf
• includes bandwidth, program-id, codecs parameters
  • see rfc4281 for codecs options
• client current starts with initial stream and works upwards, though this might change with future implementations
• really useful slide

audio only

• can include still frames in an audio only stream
• want 3:2 aspect ratio
• still frame gets included in each segment (so can change on a per segment basis)

encryption

• encrypted by segmenter
• control key access using standard HTTP techniques
• add a #ext-x-key tag, w/ method & URI
• will apply to all subsequent segments until another key tag appears
• can clear with by having a tag with method:none

streaming encoders

• supported by hardware encoders
• don’t have a software solution that Apple are recommending at the moment
• average segment duration: find 10s is a good number
  • can go shorter, but don’t want to go too short
  • means that you have to supply more segments, and your server will take more load
• have to have at least 3 segments at once available in playlist
• users can seek if you provide more

segmenter

• Apple provide a segmenter tool
• want to get the one from the iPhone dev center — it’s newer than built-in with snow leopard and has extra features

deployment

• for CDNs — want to set expire time on delivery file to be about half of segment length

notifications

• can send a highlight parameter as a notification
• app can request that highlight to get a custom playlist
• e.g. when a goal is scored…

playback

• MPMoviePlayerController only does landscape
• UIWebView can cope with portrait
• iPod app can play portrait if you make it a video podcast
• can’t have a local HTTP stream

html content

• put embed tag inside video tag so old browser will cope

references

• Nonlinear by Michael Rubin
• Digital Video and HDTV: Algorithms and Interfaces by Charles Poynton
• Simple Encoding Recipies from telestream.net

Apple iPhone Tech Talk London 2009: iPhone User Interface Design Essentials

Eric Hope ehope@apple.com — User Interface Evangelist

• solve a problem
• clearly defined style
  • axes: (usage) serious / fun; (content) tool / entertainment

serious tool

• use alignment well — see address book & settings for examples
• avoid redundancy
  • if a word is showing up more than once on a screen, get rid of it
• prioritise info, esp. above the fold

games (fun entertainment)

• games should be straight in — no hierarchy
  • most games are casual — you don’t know how long the user will have
• games should be multitouch enabled — most are
  • any that aren’t are frustrating
• lowercase “i” means go back to menu/options
• give loud visual feedback

serious entertainment

• stick to standard UI elements for navigation
  • the user doesn’t want to learn new things to get to their entertainment

utilities — dead centre on the grid

• as graphically rich as possible, on a single screen
• can they be run five feet away from you in a dock?
• equate to single-use appliances

essential

• one door to one room
• want a tree, not a web
• shared things should be in a modal sheet
• clean layout:
  • focus on structural integrity — as if they were real physical objects
  • the human mind feels uncomfortable if the UI would fall over if it were physical

gorgeous application icon

• one of the most underestimated and undervalued contributions
• legible:
  • one primary silhouette — can you do it in a shadow on the wall?
  • the mind parses shapes, then colours, then words
• high quality (fidelity) art
  • store is saturated with different icons
  • initial apps had equivalent to apple, but now full colour, hi quality
  • apple don’t need to change their apps — they’re built-in
• good example:
  • ramp champ — didn’t sell well due to icon ambiguity
  • pretty and well-defined, but name and icon didn’t mean much
• “your icon is your business card”
• make sure any branding doesn’t obscure silhouette

hi fidelity UI

• tactile design:
  • no intermediary to interface (mouse, keyboard)
  • want wood, leather, aluminium — increases perceived value of app
• free yourself by starting with paper prototypes

dynamic content — most essential

• your app needs a pulse
• apps have a short shelf life on the store
  • most mac apps are serious tools
  • most iphone apps are not — they’re content consumption based
• most users read what’s in the update before they upgrade
  • don’t just “fixed bugs”
  • even if you say “fixed memory leak that affected 1% of users”
  • it’s a “love letter to your customer”

add in-app purchases

• adds investment to your app
• people will come back if they’ve purchased content within your app
• even if a user only spends 99¢ on apps a week, they could spend 10 x 99¢ within a single app — increases their return on investment
• make sure that users can pick up their existing purchases when they get a new phone

animation

• e.g. springboard press and hold
• avoid continuous animation

sound

• “the forgotten frontier for iPhone development”
• 90% of non-game apps have no sound
• compulsion like a mother going to a crying baby
• e.g. mail sent sound in the background even when you’ve moved on to doing something else

polish

• standard alert is generally a bad thing
• icon for spotlight is 29x29 pixels — make sure you stand out
• don’t introduce push notifications until it’s solid
  • people will disable it and never see improvements
• support extras like copy/paste, undo/redo (generally uses shake)

app definition statement

• (your differentiator) (your solution) for (your audience)