Tuesday 4 October 2011

Over The Air 2011: The FT Web app - We've Got a Website for That

Andrew Betts, Assanka @triblondon

  • An app is:
    • responsive design
    • responsive to human interface (mouse, keyboard, touchscreen, TV remote, …)
  • The Daily app is now generating less than 50 tweets a day (and still going down)
    • Down from over 200
  • iPad web app also works on other devices
    • not just tablets – phones too
    • currently only accessibly on iPad
    • works on Android & QNX but not released yet
    • Android imminent, QNX a few months
  • server serves the same code to all devices
    • client does customisation and caches as much as it can
    • so can go offline and still access pages
  • FT web app
    • 8 months with 3 people
    • then additional 4 months with same people to get Android & Playbook

issues

  • flowed columns and flowing text around fixed elements
    • adobe webkit fork has great flow additions - but not available on devices
    • css3 template layouts – but again not supported
    • so have to measure content and cut it into positioned containers using Javascript
  • content balancing
    • always show a whole number of items across the width
    • done by classifying devices into four groups according to screen size
      • small, medium, large, large wide
  • podcast pages
    • want to keep playing audio even when move to another page
    • put audio player at the top of the DOM, so not altered
    • entire app is in a single page – just swap content in and out
  • swiping between sections
    • continuous carousel made out of three divs
    • middle one is always the current
    • outer ones are preloaded as required
    • implemented swipes using touch gestures
    • have to do your own gesture interpretation
      • distinguish between slow drags and flings

tools

  • playbook has remote web debugging
  • WeinRE can do similar, but can’t debug Javascript
  • TouchScroll allows you to keep headers fixed and snap to columns when swiping across
  • web debugging proxies (e.g. Charles)
  • use desktop for layout, but real device for interaction (swiping)

discoveries

  • people use native apps differently
    • they tap lightly and fast
    • the web browser waits for fast taps in case there’s a double tap to zoom
    • if you don’t want double tap to zoom
    • Assanka made fastclick

offline access

  • launching app offline
    • need to specify URLs in app manifest
    • if you change a single file, all of the manifest needs to be re-downloaded
    • store as little as possible
    • the more you store, the longer your app will take to launch
    • iOS manifest is buggy – you can get name conflicts with other webapps
      • so make sure you give your manifest a custom name
    • development is hell – things get cached the whole time
      • instead, add a comment that changes every minute (for dev)
      • still leaves you time to refresh a few times to check the cacheing
  • manifest is no good for editorial content
    • local storage is much faster than SQLite
    • so if you only access by key then use it
    • you need permission to store > 5Mb
    • strategy:
      • launch app from web, just store <= 5Mb and prompt to save to home screen
      • saved from home screen, immediately ask for 50Mb and never ask again
    • images downloaded from server as base64 encoded strings
      • can combine images into single requests
      • and can then gzip the whole thing
      • stored in local SQLite
      • rendered into DOM as a data: URI
      • base64 transfer also avoids operator image recompression!
  • analytics
    • can’t use Google analytics
    • log user actions into local DB
    • POST the log when requesting updated content
    • on the server, rewrite the log into an Apache format access log

unresolved issues

  • SSL security: no browser chrome, so no browser padlock
    • trusted, because we tell you…
  • Social media integration – OAuth in separate window doesn’t work
    • have to reload completely when return to app
  • offline adverts:
    • not technically hard, but ad server companies don’t know how to deal with them
    • can’t retract the ad
    • can’t measure clicks
    • can’t click through at all…
  • automated QA – selenium doesn’t really work
    • ended up using lots of people and lots of devices

platforms

  • Android is by far the poorest web environment when compared with iOS and QNX
    • mainly due to lack of hardware-accelerated CSS transitions
    • so FT have a native Android app which has a single native component – the gallery to enable swiping
    • the rest of the app is HTML/CSS/JS
  • QNX has a native app too, but only to have a home screen icon and to provide distribution

Q&A

  • upset Apple?
    • probably not – FT are specific market
  • differences between web app and native
    • web has more sections than native ever had
    • web app preloads content, so people swipe more often
    • as a result web app gets much more interaction
  • is not appearing in the app store an issue?
    • yes, but instead making tactical native apps
    • e.g. FT Top 100 Companies – tells users if you want the full edition, go to the web app

Over The Air 2011: OpenBTS - Open Source GSM

Will Rogers, Senior Consultant at Detica

  • Open Base Transceiver Station: http://openbts.sourceforge.net/
  • software implementation of radio towers
  • USRP: universal software radio peripheral
  • written in C++ on top of GNU Radio
  • fairly stable – maintained by Free Software Foundation
  • only acts as an access point – doesn’t simulate entire mobile network
    • another project: OpenBSC does more
  • but, can translate GSM into VOIP – Asterix
  • originally built by David Burgess (Range Networks) and Harvind Samra
    • Range Networks building commercial implementations (e.g. Femtocells)
  • originally designed for:
  • enables GSM network for $1/month per subscriber
  • hardly uses any power
  • range depends on antenna & height
    • Burning Man covered 5km – with a microwave backhaul
  • supports handset registration
    • requires no pre-provisioning
    • get a text with a code – reply and your IMSI gets added to the asterix
  • some branches support USSD (free data)
  • requirements:
    • hardware:
      • can run the whole thing in a VM
      • USB (for USRP 1), or ethernet (for USRP 2)
    • software:
      • most linuxes (Ubuntu well supported)
      • GNUradio
      • Asterix PBX
  • USRP was chosen as it was available, but it’s not really designed for GSM
    • better to have multiple of 13MHz clock
    • daughterboards available for various RF frequencies
    • need to have GSM-specific one
  • resources
  • channels
    • default is one logical channel for control
    • everything else (7) for voice
    • that means 7 simultaneous handset calls at once
      • e.g. 3 on-network conversations + one outbound
    • if you want more then you need multiple BTS units
    • if want SMS then need to steal a voice channel for control
  • SMS messages need routing, so OpenBTS includes smsqueue which forwards messages
  • limitations
    • doesn’t support live handover of calls
    • no data support (GPRS or Edge)
    • no way of supporting roaming or billing
    • 3G/UMTS boxes are available, but not yet open source
      • OpenBSC may get there first
    • doesn’t support encryption
  • use CC/MNC of 001/01 – these are the test values
  • OpenBTS console has various commands
    • timsi lists connected IMSIs and IMEIs
    • testcall creates a UDP connection to the phone
      • you can then send Layer 3 packets
    • sendrrlp sends a request for location (as mandated by US Gov)
      • can provide info about cell tower locations and phone will calculate location itself

Over The Air 2011: Exploding the Gap Between Web and Native

James Hugman - Engineer at Future Platforms @jhugman

  • Web vs Native is a false dilemma
    • the answer is: it depends, or it’s both
  • often a lot of display in UI
  • story of the glastonbury app
    • Orange app for iPhone, Android and Nokia
  • three/four phases of content:
    • before (and before embargo)
    • during
    • after
  • lots of data
    • 3000 bands, 65 stages – timings could change

experimentation & research

  • assumed offline and native UI
  • but native was too expensive for three platforms
  • how do you make native in web technologies?
    • every phone has a different web view
    • Android users don’t like seeing iPhone UX
    • and Android back button can exit app if you’re not careful
    • there is no release cycle of WebKit
      • manufacturers just take the trunk and fix bugs
    • dev and cpu time spent getting UI just right
  • Titanium: write your UI in Javascript, render natively
    • ListViews either don’t look right, or leak memory
    • doesn’t always deal with latest design patterns – can’t override platform-specific behaviours
  • Impact for iOS: javascript talks to OpenGL
    • experimental & proprietary
    • morphed into appMobi’s DirectCanvas
  • game{closure}
    • pre-alpha – HTML5
    • also node.js
    • debug your mobile app and your browser app all in the same browser
  • but Game UI is not platform UI
    • easier to make cross-platform game UI as the UI is brand-specific, not platform specific
  • important to get events passing between rendering and UI logic
    • all frameworks bundle their own Javascript engine
    • Android users don’t like big apps – Titanium hello world is 6Mb!

new platform

  • Kirin requirements:
    • native, platform appropriate UIs
    • minimal download
  • design:
    • native UI
    • JS application logic
    • native platform
  • advantages:
    • raw events processed natively
    • only app-specific events passed through to app
  • “the keyhole”
    • instantiate an invisible web-view
    • build proxies for objects
    • javascript calls into native and the other way
  • modular javascript using CommonJS
    • each screen has its own module (one file each)
  • threading using asynchronous APIs, inspired by node.js
  • native-specific features available as services
    • e.g. Location, SQL, Network access
    • same Javascript to native bridge
    • JS just uses a require(...)

releasing as open source

  • available now on github
  • as a 0.5 release
  • apache license
  • looking for feedback
  • iOS and Android only at the moment (Qt coming later)
  • Windows Phone 7 might come later

Q&A

  • testing & debugging:
    • build script runs node.js
      • so can include unit tests at build time with assert
      • have console access
      • no actual debugger
  • why port model code – isn’t it simple?
    • but if using database on device, then model not so portable (Core Data doesn’t port well)
  • porting pain? scaling to iPad?
    • most of pain was electronic programme guide
    • download & syncing

Over The Air 2011: Crap! It doesn't quite look right!

Or… how I learned to stop worrying and set my web sites free

Lyza Danger Gardner, co-founder Cloud Four @lyzadanger

  • Founder of Cloud Four in 2007
  • Explicitly for mobile web
  • Lyza was on the team that built the Obama app
  • Went through the app and realised that there was nothing in the app that couldn’t be done with the web
  • hautelook.com – high performance mobile web
  • deschutes brewery – one web
    • uses geolocation to find where to buy the beer
  • mobile web is hard and getting harder
    • and it’s not just phones any more…
    • car radios, televisions, …
  • and there’s a whole stack every time (any of which can change)
  • in the 1990s, took print ideas and applied them – pixel perfect
    • customers believed that we had control…
    • “could you move the logo a half pixel to the right?”
  • no one can possibly know enough – devices are continually appearing and you still have to deal with the old stuff
  • how do you deal with WYSIWYG for customers…?

mobilewood

  • 10 highly experienced mobile developers
  • working on where we wanted the mobile web to go
  • built http://futurefriend.ly
  • three pages, two images, no interactivity
  • 100 man hours
  • -> not scaleable!

testing is very hard

take heart!

  • relinquish control over a few things we thought we had down pat

  • four considerations:
    1. content like water
      • consider wireframes that demonstrate the flow and flex of content across a continuum of device and window sizes
      • design approach: like proportion – not pixel perfect
      • build functional mockups, so you can show (not tell)
      • it will be an iterative process
      • will not be an overnight change – it’s not a brochure, despite what we told you earlier
      • it’s often surprising how willing customers will be to adapt or modify their content APIs
      • try using textile or markdown in CMS
    2. essentials first
      • not just mobile – pare down to the essentials
      • not just about design – about process, performance, experience for every user
      • start serene and simple
    3. arm the weapons!
      • responsive web design
      • fluid media: 100% width images
      • fluid grids
      • use media queries to enhance (mobile first)
      • see also Responsive Images (by Scott Jehl)
      • Boston Globe is a showcase
      • it’s ok to leave unmanaged gutter in % widths
      • use Modernizr’s modular approach (don’t test everything)
        • but it’s not infallible
      • use server-side work:
        • reordering – moving navigation around
          • android devices are too buggy
        • reduction – smaller images
        • respect – your users: don’t download graphics if you don’t use them
      • user agent sniffing – it ain’t perfect, but it gets the job done
      • don’t be afraid to do some server-side optimization
      • optimise!
        • especially your htaccess or apache config
        • use YSlow
        • make sure you gzip everything you can (not just html)
        • cache manifests are a little tricky to deal with
      • Cloud Four images:
        • server side sizing using WURFL for major sizes
        • common images are then cached
        • then 100% fluid sizing
    4. draw the line
      • set expectations with your customers
      • explain how you don’t have control over everything
      • don’t be afraid to pull out egregious hacks!
      • be ready to fail gracefully
      • this isn’t religion!
  • frameworks:
  • iOS rotation
    • use -webkit-text-size-adjust:none;

Lyza is writing a book – Head First Mobile Web

Over The Air 2011: BBC Digital Public Space

Mo McRoberts - Developer, BBC Archive Development @nevali

  • Trying to make BBC Digital Archive accessible
  • Then working together with other organisations:
    • BFI, Kew, national maritime museum, royal opera house, british library, national archives, national library of scotland
    • have 25–30 organisations who have said yes to accessing data
    • but only have Mo to connect things!
  • lots of separate catalogues
  • each catalogue refers to things in asset stores
    • may not be able to get to asset stores, but linking catalogues by itself is useful
    • also link to external sources such as dbpedia, geonames, etc
  • want to make the archives accessible to people other than archivists
  • golden rule:
    • give everything a single, permanent URI
    • make the data about that thing accessible at that URI
  • could try to fit everything into one giant, extensible XML schema
    • or else just go with RDF…
  • can put all the RDF from catalogues into an RDF Aggregator
  • wanted to find overlaps in the catalogues
  • aggregator evaluates all info coming in and tries to find matches
    • not just exact matches, but close matches too
    • disambiguating is the hard part
  • create lots of stub objects
    • people, places, events, things, …
  • eventually want to have spindle in the hands of the public
  • the archives themselves are slowly being digitised, but it takes quite a while
  • BBC Redux captures and stores TV and radio, transcodes them and makes them available in various forms
    • been running since July 2007 for everything that’s been running centrally (not all local opt-outs)
  • now has an API and developers’ guide
  • available to developers for the duration of OverTheAir:
    • prototype RDF aggregator to query
    • API to Redux
    • references to Redux are not fully tested – may or may not work
  • genome project:
    • scan in, OCR and codify all of the Radio Times issues
    • from 1920 to 2009
    • this and /programmes will provide a public API for all broadcasts ever
  • three windows of content availability:
    1. free to air on iPlayer
    2. commercially useful
    3. out of commercial time: e.g. desert island discs (but no music)
  • aiming to have content available in 10 years’ time
    • BBC Director General has committed to this
  • 1 recent episode of Doctor Who has 80 rights clearances
    • an older episode would be worse as you would have to find the appropriate rights holders

Over The Air 2011: Stolen and lost phones - can mobile phone applications help with the problem?

Detective Sergeant Andy Williams

  • National Mobile Crime Unit – based in Notting Hill
  • Criminals stealing to order
  • Cops are not the most technologically advanced people – when Andy wants something done, he asks his children…
  • in 2001, 45% street crime linked to mobiles
    • this was from the British Crime Survey – not reported crime, but surveyed
  • current stats:
    • 9.7% of all UK crime accounted for by theft of a handset
    • 60% of mobile phones stolen in personal robberies are iPhones or BlackBerrys
      • not just the phone is high value, but the data on the phones
    • avg 8,000 phones stolen every month in london

what happens to stolen phones?

  • used until blocked (by IMEI)
    • IMEI blocking works across UK, 60% of Europe, plus North & South America coming in soon
    • but main export markets are North Africa (mainly Algeria) & Romania
  • some handsets are vulnerable to having their IMEI changed (so can be unblocked)
  • phones used for parts (broken screens, etc)
  • phones used for non-phone functions – mp3 player, etc
  • blocking:
    • operator will block SIM
    • will also request IMEI to be blocked
  • individual can install a theft tracking app, but police would need a signed warrant
    • police can’t monitor 8,000 phones a month
  • IMEIs are personal data in Germany
  • phones offered on eBay, shown as blocked or no signal

immobilise – national property register

http://immobilise.com

  • if device is registered with immobilise, police don’t need to go through RIPA
  • can put anything with a serial number on there
  • can check on people’s phones when stop people on the street
  • private company set up and operated in conjunction with the police

formation of NMPCU

  • initially front line - reducing robbery
  • expanded to all acquisitive crime
  • working at all levels of crime - local through to organised
  • now an offence to offer to reprogram a phone (change the IMEI)

Police Central eCrime Unit

  • trying to combat cyber-criminals
  • looking at denial of service attacks, etc
  • front line police notoriously poor at dealing with cyber crime
  • but no law enforcement agency that specialises in all areas of cyber crime
  • most expertise is in the forensic area
  • judicial system is not set up to deal with cyber crime
  • judicial boundaries don’t help – if online connections are not in UK…
  • recently starting up new kind of officer: Covert Internet Investigator
  • not very many across the country

Case study: organised criminals targeting music events

  • following same bands the whole time
    • bands with mosh pits…
  • Download 2010: 3 people found with 140 phones buried under their tent

Security & new mobile tech

  • Man City gave 20 season ticket holders their tickets on their phone
  • if someone steals your phone, they have access to your wallet, your ticket, your flat…
  • NFC requirements requested by police:
    • PIN required for all payments – not gonna happen!
      • at the moment, 5 payments of £10 until a PIN required
    • capture IMEI during transactions
      • possible but not gonna happen
    • compulsory registration with financial institution
      • may happen

key areas

  • registration at point of sale
  • carphone warehouse register 25,000 phones a day at Xmas
  • only 4 in 10 PIN lock their phone
    • significant barrier
  • locating devices post theft
    • works ok, but not great – won’t work if someone goes into a block of flats
  • identification of a SIM change post theft

checkMEND

http://www.recipero.com/checkmend

  • can find out if an IMEI is stolen
  • some operators run this service for free
  • but otherwise may cost £1 a shot