Thursday 12 November 2009

Apple iPhone Tech Talk London 2009: Preparing and Delivering Video for iPhone

Eryk Vershen evershen@apple.com — Software Evangelist

This presentation went really fast with loads of detail. I captured some of the highlights below, but you’ll want to get hold of the slides to get the full detail. Supposedly Apple will be putting them up later this month…

  • challenge is to deliver better pixels, not more

there are absolute upper limits

  • 1600 Kbps
  • 48 KHz
  • stereo
  • can have alternate audio tracks (e.g. for different languages)
    • under the user’s control, not yours
  • can do subtitles — but no apple-provided tool
    • can also have alternate subtitle tracks per language
  • closed captioning not available to your code
    • anyone know the

containers

  • streaming: .ts — MPEG-2 Transport Stream
  • otherwise four other containers like .mov, .3gp

resolution & aspect ratio

  • iPhone video always takes over entire screen
  • can fit or fill video with different aspect ratio
    • fit: letterbox or pillar
    • fill: crop
  • don’t support watching movies in portrait since lose too much of screen

bit rates

  • bandwidth is not a constant
  • so would want to deliver multiple versions, using adaptive to switch during playback

preprocessing

  • there isn’t a recipe…
  • big three:
    • deinterlace
    • crop/scale
    • smoothing — simplify the image

encoding

  • make reference movies so can serve up multiple bitrates
  • quicktime 7 pro includes advanced MPEG-4 settings
    • ensure you use the baseline profile

progressive download

  • structure movie for more random access
  • map of movie moved to the beginning
  • server needs to support byte-range requests
  • works with reference movies — picks appropriate movie based on data rate
    • once the system has chosen a movie, you’re committed
    • http streaming allows you to change during a movie

streaming

  • http streaming introduced with snow leopard and iPhone OS 3.0
  • need a media encoder from someone else: Apple don’t provide one
  • segmenter:
    • converts stream into segments
    • continually updates playlist file
  • client:
    • downloads playlist
    • downloads segments
  • format:
    • MPEG-2 TS
    • also supports elementary audio
  • playlist is an extended M3U file
    • should be unicode UTF-8
    • use extension .m3u8
    • #ext-x- tags added by apple to support dynamically changing playlists
    • #extinf tag comes before each segment url and indicates its duration
  • on demand playlist just like live playlist, but has a #ent-x-endlist tag at the end
  • player currently seeks to segment boundaries

variant streaming

  • variant playlist contains references to other playlists
  • just another m3u file
  • new tag: #ext-x-stram-inf
  • includes bandwidth, program-id, codecs parameters
    • see rfc4281 for codecs options
  • client current starts with initial stream and works upwards, though this might change with future implementations
  • really useful slide

audio only

  • can include still frames in an audio only stream
  • want 3:2 aspect ratio
  • still frame gets included in each segment (so can change on a per segment basis)

encryption

  • encrypted by segmenter
  • control key access using standard HTTP techniques
  • add a #ext-x-key tag, w/ method & URI
  • will apply to all subsequent segments until another key tag appears
  • can clear with by having a tag with method:none

streaming encoders

  • supported by hardware encoders
  • don’t have a software solution that Apple are recommending at the moment
  • average segment duration: find 10s is a good number
    • can go shorter, but don’t want to go too short
    • means that you have to supply more segments, and your server will take more load
  • have to have at least 3 segments at once available in playlist
  • users can seek if you provide more

segmenter

  • Apple provide a segmenter tool
  • want to get the one from the iPhone dev center — it’s newer than built-in with snow leopard and has extra features

deployment

  • for CDNs — want to set expire time on delivery file to be about half of segment length

notifications

  • can send a highlight parameter as a notification
  • app can request that highlight to get a custom playlist
  • e.g. when a goal is scored…

playback

  • MPMoviePlayerController only does landscape
  • UIWebView can cope with portrait
  • iPod app can play portrait if you make it a video podcast
  • can’t have a local HTTP stream

html content

  • put embed tag inside video tag so old browser will cope

references

No comments: