Mobile@Scale 2016

Last month Facebook invited mobile developers into their London offices for a collaborative discussion on scaling mobile development.

The focus was mainly on native development — and the attendees were mostly iOS and Android developers — but the scope expanded to include scaling development processes as well as how to scale apps for lots of users.

Jim Purbrick @JimPurbrick, Engineering Manager in Facebook London’s office, introduced the talks by saying that on mobile, the bug stakes are higher — once a bug is released, the app is on people’s phones and is much harder to fix. You can’t just update the code and see people get the fix in the next page refresh.

And for all the focus on ending up with a native app in the platform-specific app store, two of the big themes were sharing code across platforms and being able to make quick changes to apps that were already deployed.

I was impressed by the inclusivity of the conference — not only were the speakers from a variety of companies (not just Facebook or Facebook partners) but the audience were encouraged and given time to ask questions and discuss with the speakers.

The talks and discussions were all videoed and I’ve linked to them below together with my notes highlighting the points that made an impression on me.

Scaling iOS @ Google

Michele Aiello @micheleaiello, Tech Lead Manager on the Calendar app, Google

Michele gave a really detailed talk about how the iOS teams at Google deal with handling large amounts of code shared amongst many geographically spread developers. There’s lots of useful nuggets in here — and it’s interesting to see where Google have invested time and effort in order to make cross-platform and large team development easier.

Scaling iOS @ Google - Michele Aiello

Posted by At Scale on Monday, 28 March 2016

• Google has more than 60 apps in the Apple App Store!
• iOS devs have moved from London to every office worldwide
• strategy on how to share code really important
• design & ease of use crucial for scaling apps
• yearly gathering of all mobile developers
  • often start shared efforts there
• regular tech talks in local offices + recorded & streamed around the world
• have feature specific “Tiger teams”
  • one goal: ship a feature
  • cross functional: Android, iOS, web, PM, UX, API, etc
• have trouble with merging & reviewing xib, project & storyboard files
  • so Chrome team developed GYP: JSON for structure & dependencies
  • GYP: “generate your project”
  • use storyboard & xibs for prototyping, then shared code for elements
• release management:
  • regular releases every 2-6 weeks
  • with feature flags to toggle new features
  • compile time flag during initial dev, then runtime flags for later
  • known as the release train
  • heavier-weight trains need synchronising with marketing etc — ok to be a little late
  • 75%-80% of users are using auto-update so automatically get updates
• testing:
  • XCTests for functional and performance
  • KIF & Earl Grey for UI tests
• sharing code
  • single repo
  • HEAD is always stable
  • all code is available and shared
  • making a change in shared code: see the test results and roll back if issues
• for any shared code
  • enforce documentation & example code
  • catalog app for UI elements
• for cross platform sharing
  • try to share model cross platform and to server too
  • makes offline easier
  • have tried webviews & javascript
  • now using C++ & J2ObjC
    • C++ easy on iOS, complicated on Android
    • used in Chrome
    • j2ObjC used for Inbox
  • j2ObjC even lets you debug into transpiled Java code in Xcode
    • breakpoints, stepping, variable values all work
  • if code is simple, look at sharing the tests rather than the code
• swift at Google: currently playing with it
  • have found that development is faster
  • probably waiting a few months to bring into production apps
• user testing using beta releases (testflight, android beta)
  • metrics in the app
  • surveys after testing
• have tools to search whole repo to find out if code is still used
• sharing layout
  • done using sharing layout logic

When mobile IDEs need to scale

Al Sutton @AlSutton, Facebook

Al talked about how Facebook builds Android apps, and how they feed back improvements to their build process into the open source community (e.g. IntelliJ community edition and the Buck build tool). By using Buck, they allow their developers to choose whatever IDE they want.

When mobile IDEs need to scale - James Pearce and Al Sutton

Posted by At Scale on Monday, 28 March 2016

Nuclide

James Pearce @JamesPearce, Head of Open Source at Facebook

James continued from Al’s Android introduction to talk about Facebook’s new Nuclide IDE for building iOS apps… It’s exciting to see some competition in the iOS IDE world — whilst Xcode is great at some things, it often leaves a lot to be desired. JetBrains’ AppCode is a useful challenger but to have an extensible open-source IDE for iOS could be a game changer. The only downside for me is that Nuclide relies on Buck, so you have to change your project to buy in to the Facebook toolchain. Perhaps if someone could create a Fastlane plug-in…?

• unlike IntelliJ, Xcode is not open source, so can’t contribute
• existential issue for Facebook…
• started extending Atom from github
• aded Flow, Babel, Clang & Buck
• created Nuclide
• also added Chromium dev tools to help debug into app
  • lets you debug into Javascript, Objective C, etc all in same place
  • transpiling keeps source maps to help with line numbers
  • also lets you inspect into UI hierarchy for ReactNative apps
  • includes highlighting
• now have 2/3rds of committing engineers using Nuclide
• have analytics built-in
  • tracking feature usage
• internal infrastructure team has become a product team
• now available at http://nuclide.io
  • analytics kept for internal
• other open source projects
  • pop: iOS animation library
• doesn’t have refactoring yet

6 lessons learned scaling mobile at SoundCloud

Next up were a couple of sessions from smaller companies (though still not small!) showing how they built and adapted their apps faster to keep up with demand. SoundCloud spoke about using ReactNative (more on that later) and how they structured their dev teams to include mobile developers.

Jamie McDonald (Android) @jdamcd & Matej Balatic (iOS) @skavt, SoundCloud

6 lessons learned scaling mobile at SoundCloud - Jamie McDonal…

Posted by At Scale on Monday, 28 March 2016

• building out new SoundCloud Pulse app for people creating sounds
• most engineers busy on main SoundCloud listener app
• got a partner for Android, but built iOS app with ReactNative using web developers
• shared design & feature set across platforms saves a lot of time
  • were previously designing features twice
  • marketing was more complex too
• mobile specific API
  • mobile-specific features: background sync, batch fetches
  • “back-end for front-end” idea from ThoughtWorks
• developed C-based mobile playback library (skippy)
  • initially for Android, now rolled out across iOS too
  • e.g. optimise streaming for emerging markets
• tried to spread mobile devs through feature teams
  • but spread too thinly
  • weren’t able to pair and share knowledge
• instead created clusters of feature teams
  • mobile engineers shared amongst each cluster
  • could be in enough numbers together
• release train model
  • each feature team can take responsibility for shipping
  • allowed action but also feedback and responsibility
  • use feature flags — team responsible for turning on when ready
• tools used:
  • iOS:
    • FlipTheSwitch
    • stable CoreData stack — specific use of framework
  • Android, use LightCycle (soon to be open-sourced)
    • forward life cycle events to small independent modules
    • receives callbacks but doesn’t need to know which activity its attached to
    • enables better unit tests as can separate things out more effectively

Backend-driven native UIs

John Sundell @johnsundell, iOS Developer, Spotify

Spotify have an almost completely content-based app and are constantly tweaking to change the presentation and priority of different music. John and his team came up with a way of handling that change by controlling the whole app UI from the backend API.

Backend-driven native UIs - John Sundell and Diego Cristina Ca…

Posted by At Scale on Monday, 28 March 2016

• define components in backend API
• generalised data binding
• generalised components
  • implement standardised components which can be picked up from API
• can put cacheing, and lots of standard stuff in the generalised app
• control the UI from the backend
  • API contains view models rather than raw models
• Ed.: makes sense if you have an app with lots of similar components
  • especially for a content-based app
  • similar to Google’s code-based component library
• were able to delete 20,000 lines of code on home page browse view
• overall have been able to delete 100K lines of code across iOS & Android
• use layout traits to control layout
  • e.g. full width, separator, stackable
• request sends a lot of data about the device to the backend
  • can return different components & layout depending on device or screen size etc
  • sometimes send extra data in response so can handle quick changes e.g. screen rotation
• support infinite scroll using metadata with URIs for follow-up pages
• can set up fallback components — if this not available, fallback to previous
  • enables playing around with new features & UI but still supporting older builds

Infer: Moving fast with static analysis

Dulma Churchill, Software Engineer, Facebook

Taking up Jim Purbrick’s challenge of dealing with the higher stakes of bugs in mobile, Dulma gave us an introduction to Infer — Facebook’s static analyzer that can check for memory and resource leaks and null pointer issues each time you compile.

Infer: Moving fast with static analysis - Dulma Churchill

Posted by At Scale on Monday, 28 March 2016

• static analyzer that doesn’t require pre/post conditions
• compositional, so doesn’t need to process whole project at once
• very intertwined with compiler
• infer can find inter-procedural bugs not local to single file
• used with CI can be set up to only process newly compiled files
• within facebook: fix rate around 70% in recent months
  • high rate due to getting results on continuous integration
• there’s an Xcode plugin
• integrated with codeboard
  • web-based IDE to teach programming in classroom
  • Java, Python Haskell…
• see their blog post about being used at Spotify

3000 images per second

Henna Kermani @tokyotwilight, Software Developer, Twitter

Some interesting stats from Twitter here, in Henna’s story of how Twitter scaled up their image and video handling.

3000 images per second - Henna Kermani

Posted by At Scale on Monday, 28 March 2016

• image uploading used to be all in the same API call as the tweet itself
  • any point of failure would fail whole thing
  • waste of bandwidth for client & server
• split out image upload from tweet content
  • also allowed segmented, resumable uploads
  • used multi-part POST requests with separate INIT, APPEND & FINALIZE API calls
  • massive drop in upload failures, especially in developing world
• did research on age of images:
  • 15 days 50th percentile
  • 150 days 90th percentile
• so kept original + 20 days of variants
  • balance between storage increase per day and computation on each request
  • saved $6m in 2015 just from this change!
• image formats
  • tried using WebP for 6 months last year in Android app
    • ~25% smaller than PNG or JPG — better engagement
    • but not supported on Android <4 or iOS…
  • converging on progressive JPEG instead
  • used Facebook’s Fresco library in Android app

React Native

Pieter De Baets @javache, Facebook

Pieter gave a detailed intro to React Native — building native apps for iOS and Android using just JavaScript and HTML-like markup.

React Native: bringing the best of web development to native -…

Posted by At Scale on Monday, 28 March 2016

• if you ship a bug in a mobile app, there will always be a user out there running that bug — no matter how many updates you apply…
• write UI declaratively, code in Javascript
• share lots of code between iOS and Android
• Apple’s guidelines don’t allow you to update code in a running app
  • but there’s an exception that lets you update JavaScript over the air
  • so you can update React Native apps instantly

Don’t forget the web

Jeremy Keith @adactio, Founder, Clearleft

After all that talk of native development, Jeremy brought us back to thinking about the web and how it will always be the largest, widest target. It isn’t a “platform” and it will never be the leading edge of mobile, but it is for everybody.

Don’t forget the web - Jeremy Keith

Posted by At Scale on Monday, 28 March 2016

• when building for the web
  • start with core functionality
  • implement with simplest techology
  • enhance!
• can be done for whole service but also for individual components
• Ed. is this that much different from native?
  • especially for different OS levels, Android features etc
• there’s always something new that’s not fully supported

MomoLondon: Cloud Computing & Mobile

A new venue tonight — up on the 9th floor of Google’s Central St Giles office. They even provided good food for the hungry mobile mondayers…

The evening’s focus was on Cloud Computing and Mobile, and the questions seemed to focus mainly on the cloud with not much discussion about how it related to mobile. The chair, Camille, kept the discussion moving along briskly, but we didn’t really get into any depth on any of the subjects. Maybe the audience could smell the food!

The panel

• Chair: Camille Mendler @cmendler
  • Principal Analyst at Informa
• Rob Easton @Cloud_Easton
  • Head of Google Cloud Platform UK & Ireland
  • Eurovision app used App Engine for back-end
• Dr Janko Mrsic-Flogel
  • CEO of Private Planet — personal cloud
  • 7 out of 10 carriers are providing a personal cloud solution
• Caroline Van Den Bergh @GoldenGekko
  • Golden Gekko mobile agency
  • 85% of apps we build are based on Google App Engine
• Jonathan Raper @MadProf
  • CEO at Placr/transportapi & Visiting Professor at City University
  • purchasers of cloud services
  • mostly built on memset

outages & availability

• JR: best service providers are those that communicate well when things go wrong
  • one provider went down and didn’t tell us anything for an hour

looking forward

• RE: google datacentres include a new one in Taiwan at 15 hectares
  • green computing
  • Google can provide very low latency
  • enables reaching out to new business areas such as the film industry
• JR: still talking quite aggregated performance measures
  • can’t ask for X many transactions for Y pounds
  • not that detailed
  • next step may be platform products
• CVDB: mobile enterprise software saving two days a week
  • managing remote teams
• JR: sometimes blame is shifted between telcos & cloud
  • would like to have better tracing
• JMF: Private Planet have a totally green datacentre in Telecity in Holland for T-Mobile

internet of things

• JR: the problem is trading information
  • automatically discovering different feeds & blending them
  • spend 3 weeks integrating code and 3 months arguing over terms & conditions
• CVDB: best successes have been smaller ones
• CM: find my yak in nepal
  • the yak’s don’t talk back — so there’s no privacy issues!
• RE: getting electricity usage feeds every 10 minutes
• CVDB: there has been scaremongering around Google capturing data
• JR: all our data is royalty-free downstream
  • makes it much easier to deal with
• CM: Copenhagen have bike devices to share traffic information
  • provide individual exercise info
  • but also shared with municipality
• JR: Waze has lots of complicated terms & conditions
  • data is available but locked up

the next generation

• CM: has been proved conclusively that 3D gaming improves your cognitive ability
• JR: …minecraft
• JR: it’s still a bit like early 2000 — mobile operator walled gardens
  • want to take down the barriers and bring the data silos together
• CVDB: working with Saga
  • older people have more time, want to delve deeper
  • tablets more popular than mobile phones

data scientists

• CM: shouldn’t we moving towards everyone manipulating their own data
• JR: digital literacy is very shallow, especially in highest decision-making tiers

security issue — data protection

• safe harbour in tatters
• can’t use the cloud at all if we have any secure data
• JMF: Private Planet always has data centre in the specific country
  • but are governments the biggest problem? what about the corporates?
• JR: if you need to do it, in-country encrypted solutions are available
• RE: sure, there are some workloads that Google can’t take on
  • e.g. we won’t let you audit our data centre
  • so can’t work under FCA regulations
  • over time, some of these regulations are going to change
• JR: all have to do as much as we can to help the government make the right decisions
• JR: sometimes the safest place to do something is in the cloud — e.g. running Windows
  • we don’t allow it on any of our machines
  • but it’s great in the cloud!

what if cloud provider company goes bankrupt?

• JR: escrow
• JMF: most agreements done via a channel partner — so that partner will continue the service
  • single party relationship is much more complicated
• RE: cloud business could be bigger than ads for Google
  • SLA penalties are often a percentage of what you pay
  • but if you pay less and less, the SLA means less and less…

is it becoming like supermarket wars?

• RE: Google reduced prices — and if you use an instance more than 25% per month then get a discount
  • terabyte of storage is $26/month
• CM: just like a loyalty card…
  • who will be Lidl and who will be Waitrose?

will we still be waiting in several years?

• JR: overestimate change in the short term and underestimate in the long term — change is stepped
  • sometimes have to wait for the step change in technology or regulation
• CVDB: business are going to grow up quickly
  • first digital generation CEOs will be around in the next few years

third party dependencies

• JR: Apple reviews can be capricious
  • if Apple fall out of love with Dropbox, then that could cause a problem for lots of apps
• JMF: was a huge debate on mobile medical health a while ago
  • then suddenly there was an app that just works
  • and the issues can just disappear
  • gadget-led terms and conditions…?

speed & access?

• RE: check out server density review
  • blog by someone who reviewed lots of providers and switched to Google
• CVDB: Golden Gekko mandate that all backends must be on Google App Engine
  • mainly so that clients can scale

new innovations in infrastructure?

• RE: Google build our own servers
  • apparently they would be the 3rd biggest manufacturers in the world
  • they value new innovation

where is the next growth to make it easier?

• RE: focused on the developer
  • acquired StackDriver to manage cloud capabilities
• JMF: If This Then That
  • making cloud development open to the consumer
  • certain mobile operators are understanding this and joining up devices in the home
• CVDB: finding that parse.com doesn’t suit
  • one size doesn’t quite fit all
  • prefer google’s flexibility

inviting consumers to get more involved

• CVDB: user testing
  • avoiding ivory tower
  • developed O2 Priority Moments
  • built packaged solutions based on it
  • incentivized behaviour also used in enterprise apps
• JR: designed a first location bank
  • your own location storage — that you can choose to share
  • (Ed. see also fireeagle)
  • next generation have a different approach to privacy
• RE: snapchat is a massive google customer
• audience: teenagers using technology completely differently
  • using snapchat to have a whole conversation
• RE: snapchat consumes more IT than any banking industry
  • JDI has 10 petabytes on Google cloud
  • consumers of IT will be well away from the enterprise
  • can have failure at a much cheaper cost — can learn more

what financial services are possible in the cloud?

• RE: restricted by FCA regulations, but other options possible
  • e.g. datasift to search across twitter
  • big challenge to attract IT talent in financial services as there are so many restrictions

skills

• JR: right now it’s hard to hire — ruby on rails developers are hard to find
  • constraint on development right now
• CVDB: couldn’t agree more — mobile skills are in short supply

tools for analytics & loyalty/crm

• looking at how data can be analysed
• wanting to drive revenue per user
• wanting to get the right tools in place to complete the feedback loop
• CVDB: iris solutions company
  • built the backend themselves
  • ?? http://www.iris.co.uk/
• JR: “build it in the cloud, but just don’t tell IT”
• RE: tableau, BIME, etc. working on top of Google Big Query
  • now a lot cheaper

After the panel discussion

As often happens with Mobile Monday, the discussion after the panel was richer than the panel itself. Some discussion points that came up when I was talking to people afterwards included:

• How do you balance keeping data on the device and in the cloud, to deal with intermittent network availability?
• Will the banks get clobbered by more agile personal finance companies that don’t have to worry about so many regulations?
• Why has it taken Google since 2008 to move from basic App Engine web apps to a full cloud platform?
• How will Google make more money from Cloud Computing than from Ads? There’s a big market, but the price is dropping…
• …and on a completely different note, several people were interested in setting up Code Clubs in their local primary schools, but were stretched for time — perhaps they could work together and contribute part-time?

Droidcon 2013: Teaching Your Client Android Design; or, Ridding the World of iPhoneys

Kenton Price, Little Fluffy Toys http://www.littlefluffytoys.mobi

Presentation available online

Summly

• Android has algorithm on client as Android screens come in different sizes
• insisted that wanted back button in app
• got google developer advocate support to say no
• also added various improvements, several of which

BBC Weather

• again received an iPhone design from the agency
• identified areas that should be more Android-y
• added additional Android features
  • NFC support — share locations over NFC
  • lock screen widget, daydream mode, dashclock integration
• don’t need to shout about these integrations, as early adopters will do it for you
• first release: got slated on play store as there was no landscape mode
  • tablet users will review you (no easy way to exclude…)
  • upset users are far more likely to leave reviews
  • designers hadn’t included landscape as it was not seen as a big use case
• about > version code > tap 7 times > easter egg :-)
• reviews: “BBC does it better on Android for once”
  • made design Android native
  • then added nice Android features
• even the BBC were surprised about what could be done

Play Store beta management

• assign a Google Plus community to different (alpha/beta/etc)
• accept/reject membership
• LFT used to use HockeyApp, but this is much better

how to

• get the product manager on board
• get graphic designers on board
• compare to best-of-breed
• get the ActionBar absolutely right

Q&A

• some designers feel that they haven’t designed it if there’s still native components visible
  • a lots of designers got burned by the change from iOS 6 to 7
  • maybe they’ve learned their lesson now!
• supporting older Android
  • support 240x320 as the smallest, but was difficult to work around lack of bucket design sizes (3+)
  • couldn’t test across everything
  • used ActionBarSherlock
  • lots of graceful degradation

MomoLondon: Mobile, Maps & Geolocation — so much more than “Where am I?”

This Monday’s session was a little less lively than the previous session on mobile marketing. I expected the panel to violently disagree with each other on licensing, data costs and providing data to the public… but they were all far too polite!

The panel discussion covered a wide variety of subjects, fulfilling the session’s premise: geocoding, open data, data quality, visualisation, alternative methods of navigation, indoor location… It was a huge area to cover in a couple of hours.

For me, the main change in the last year or two has been the opening up of data sets. Open Street Map has gone from strength to strength and can now be linked up with live transport information and other citywide data sets.

As Jeni Tennison remarked, the most exciting area at the moment is putting all this data together in innovative ways. More than the Google Maps mashups of earlier years, we can now experience 3D immersive visualisations of cities, with real-time data pulsing through the streets and under the ground — like ViziCities, for example.

Panel

• Gary Gale @vicchi
  • Director of Global Community Programs for Nokia’s HERE
• Christopher Osborne @osbornec
  • Currently at AlertMe, and founder and organiser of GeoMob
• Jeni Tennison @JeniT
  • Technical Director at the Open Data Institute and member of the W3C TAG
• Ian Holt @IanHolt
  • Head of Developer Outreach at Ordnance Survey
  • just launched their new OS OpenSpace SDK for iOS
  • offline maps really useful
• Harry Wood @harry_wood
  • Currently at http://placr.co.uk and OpenStreetMap volunteer

How easy is it for devs to turn lat/longs into human-readable info?

• HW: hard work to go to postcode or nearest POIs
  • getting hold of open data is important
  • is displaying the map the best thing?
  • FourSquare did really well with a location-based app that didn’t show a map
• GG: location powered by FourSquare has become a heavily used API
  • it’s free to access but large amounts of usage lead to business negotiations
  • similar for other geocoding APIs
• JT: Open Street Map allows us to choose what we want to mark on the map
  • Ordnance Survey was mainly restricted to Churches and Pubs…
  • Open Street Map has shops (and pubs!)
• IH: how do you ensure that all the data is there?
• HW: it’s a problem that goes away with time
  • always find your nearest pub? getting there in the UK, but some patches in the north with less coverage
  • but the map gets built out on an interest-first basis, so better coverage in more interesting/populated areas

How much do we trust what we see on our mobiles?

• GG: in US a woman sued Google ‘cos the walking maps told her to walk across all 16 lanes of a superhighway…
• HW: we trust it too much in some cases
  • OS really shines with quality of data
• IH: Google & Apple’s branding has gone a long way to encourage trust in map data
• CO: Apple chose not to visualise lots of data on the map, but they got panned for it
• HW: there’s an active market in insuring map makers against maritime mapping mistakes
  • it’s an expensive mistake when a supertanker runs aground…
• GG: people have been hauled before the coastguards for using consumer mapping systems to navigate ships
• GG: people tend to accept what they have on their phones

Should map data producers make end-user systems? Or leave it up to smaller companies?

• IH: paper-based maps couldn’t change, so had to make one-off presentation decisions
  • now the data can change for the user context
• CO: routing is hard - still dominated by big companies with background with navigation
  • but lots of space for smaller companies in other areas
• IH: huge explosion of neo-geographers a few years ago
  • lots of mash ups and visualisations using geo data

What’s next in display? Will wearables move away from the concept of maps?

• IH: we’ve always been shifting display methods…
• Jo Rabin: years ago, a taxi driver in Chicago had a book of turn by turn directions but no map
• HW: a large proportion of people get rather upset when they see a map (including Harry’s girlfriend…!)
• Audience: worth thinking about how blind people navigate
• IH: navigating a city by sound — turn away from the areas that sound threatening
  • Ed.: this is not new — there was an MIT project in this vein in 1999!!
• HW: Google Street View seems a useful addition at the moment, but not a replacement for basic navigation
• CO: 3D mapping is a bit gimmicky at the moment, but the data gathering behind it is massively useful
• GG: for every mapping service, you only ever see less than 5% of the underlying data
• CO: one of the best designed apps for mapping was an app for blind people on old Nokia phones — vibrate once for left, twice for right

What’s new and exciting in the next 3 years?

• GG: indoor navigation
  • multiple companies working on it
  • but all currently incompatible
  • holy grail: directions from your house, through the tube to the exact exhibit in the V&A Museum
• JT: putting open data together, e.g. being able to know when you’re in an area with a high risk of flooding after rain
  • 3D sim city-style simulation of London, with data such as crime rates superimposed
  • how can we bring in time-based information to our space-based navigation?
• IH: early years education
  • inherent part of the general data structure
  • Ed.: Scratch, a programming environment for children, is working on integrating Google Maps
• IW: Ed Parsons (Maps @ Google) — “geospatial will no longer be geo-special”
  • not just a map screen in the app, but mixed throughout the app
  • promise of access to raw data
• GG: NOAA have produced a low-power app which warns you about tornados, just using cell tower data
  • another innovative location-based service which doesn’t require a map…
• CO: massive growth in transactional apps, e.g. Hailo & YPlan
  • also Google etc building an ecosystem e.g. their purchase of Waze
• GG: wearables: I wouldn’t buy Google Glass, but I would buy a smart watch
• GG: hospitals making savings by tracking their equipment locations (and stopping it sprouting legs)

Licensing

• GG: businesses scared of using OSM data due to fear that they will have to share their private data
• HW: share-a-like license aims to ensure that data is not taken and locked away
• JT: OpenCorporates make company data available for free on a share-a-like license, but also offer a paid license with less viral terms
• HW: OSM can’t do that — would have to agree with all contributors…

What about mobile operators? Indoor location?

• GG: cell tower location works well in urban areas
  • less so in countryside…
  • and mobile operators in Germany are restricted to telling you only two of the nearby cell towers (not enough to get a fix)
  • mobile operators had their chance 8-10 years ago when they tried to sell their data to marketing companies
• CO: starting to see traction in government
  • e.g. who’s moved on what roads at what speed
  • useful for real-time traffic management
  • high value areas: marketing & business intelligence

Coping with geospam…?

• HW: Google are already dipping their toes in the water there (Ed: e.g. the map tiles have advertising built-in)
• IH: walking past a shop and being offered a service
• GG: been proved to be pretty worthless — you’ve already past when the message comes through
  • data of where you are not so valuable
  • where you’ve been is more valuable
  • where you will be is where the money will be made

Apple iBeacons — Bluetooth LE with an ID

• CO: one inspiration for that comes from certain countries with restrictive regimes
  • walk into a bar and turn bluetooth on — get offered services that are normally not available
  • need a strong well-designed way to broker your data

Where are you failed by current map services?

• HW: developing world — none of existing providers have coverage
  • Google Map Makers pushing hard
  • Open Street Map
• Dan Appelquist: cycle maps for London
  • Google maps keeps on directing me through routes on Hampstead Heath that are not passable by bike
  • http://cyclestreets.net routing for bikes based on OpenStreetMap data
• CO: pedestrian mapping
  • most of the data has come from driving cars around
  • huge value as people much more likely to make a purchase when they are walking
• GG: but getting data is hard as many areas are private
  • no public right of way
  • hard for commercial companies to map
  • OSM has an advantage here — can’t really sue OpenStreetMap for trespass

How long rather than how far away?

• IH: common GIS function, but doesn’t crop up in consumer stuff
• JT: mapumental by mysociety — where should I live so that I can commute to work?

Royal Mail PAF file restrictions

big database of postcodes & business locations for the UK — really valuable and the Royal Mail are holding tight…

• JT: campaigning to open it, but it seems extremely unlikely
  • would have to create an open street gazetteer
  • if someone would like to sponsor it — please see Jeni!

Linking to map model data

• HW: most OSM data has a URL
• GG: for roads — Nokia has Routing Segments, OSM has Ways, OS has ITN (Integrated Transport Network)

Summary

• JT: collaboration is the most interesting area
• IH: everybody should edit something on OpenStreetMap.org
• GG: expected a question on location & privacy, Facebook & Google…
  • “I now trust big internet companies more than I do the government”

Next session

• July 15th: Mobile Operating Systems — with Ubuntu from Canonical

Google I/O 2011

Some brief notes on the Google I/O 2011 keynote this afternoon… It started kind of dull with Android stats that were pretty good but still seemed to be catching up on Apple, then gradually got better and better.

Google really does want Android to be “one OS everywhere” (I guess running “one web” webapps…) and the UI is improving. I look forward to the media content coming across to countries other than the USA, though the lack of any mention of music or movie labels leaves me in some doubt as to when this will happen. The presentations neatly glossed over the time involved in syncing music and movies between devices and the online service — even on a home wired network the data would take a long time to upload. And there was no mention of syncing between devices on the same LAN like Dropbox either.

The big news for me, however, was the accessories and android@home. By making it so cheap to add hardware integration with Android devices, there could be a real explosion of new products and the home automation market could finally break out of eye-wateringly expensive or home hacker territory. I’m really looking forward to more detail on android@home, but there’s nothing released yet…

android updates

• widgets can now be stretched
  • scrollable -> resizeable (just change XML)
• android can now be a USB host
  • keyboards, mice, game controllers
• google tv market open for devs this summer
  • will be based on android 3.1
• icecreamsandwich coming Q4
  • advanced UI
  • APIs to help scale UI
  • action bar able to reconfigure itself for available space

media

• movies and books now available through android market
  • can watch on web or on android device (e.g. android TV…)
  • seems to be US only (like Music)
• google music
  • cache music recently played
  • can “pin” music just like on movies to make available offline
  • downloads in background
  • 20,000 songs can be added to library
  • free in beta…
  • new music app works with android 2.2 and above

hardware

• new devices will receive latest updates for 18 months after first launch
  • (if the hardware allows)
  • guaranteed to come through reasonably quickly
• android open accessory
  • plug in an accessory
  • phone gets taken to app (or to android market to download it)
  • supports USB now and bluetooth in future
  • hardware design toolkit based on Arduino
• android @ home
  • android @ home framework
  • new low cost connectivity protocol for non-wifi devices
  • why not existing??
  • partnering with several industry players
  • LED lights & switches
  • tungsten demo with NFC chips in CDs cannot work unless Google get record label deals…

new devices

• Samsung 10.1 tab: 720p, 565g, 10.1”
  • lighter than iPad 2, at least as powerful…
  • just waiting for better interface

Londroid: Payment and monetization

The first of two Android blog posts — this one’s from the Londroid Meetup on April 21st themed on Payments.

There were three presentations from three competing payment platforms, and they presented in roughly the order I’d prefer to use them (in their current state): Paypal is very much sorted for Android — they make it easy and it looks impressive and secure; Google’s in-app billing is still new and feels unfinished; and Paythru is simply a browser-based system that’s designed for mobile, but no way near as slick as the other two.

However, talking to the Paythru rep later, they have some interesting tricks up their sleeve — like being able to take payment but not actually charge it to the card for several days (useful for synchronising multiple payments, or buying Olympics tickets). Google will also be beefing up their billing system over the next few months, so Paypal won’t have an easy ride.

Integrating Paypal In-App and Mobile Browser Payments on Android

Anthony Hicks anthony@x.com — Paypal

• paypal in EU biggest in UK, then DE, then FR
  • small numbers in scandinavia, spain & italy
• now have a bank license in Europe
• iPhone, iPad, Android mobile payment library
  • all in-app (no browser)
• adaptive payments
  • split payments between multiple parties
  • make chains of payments
  • make a billing agreement and charge them later
  • pre-approval — can request payments without taking people to paypal
  • customer agrees to pay up to £X per month
  • take commissions on payments
• mobile express checkout also available (browser based)
  • optimised for webkit
  • can login with mobile number and pin (as well as with email/password)

Development

• payment button & pages have to be consistent
  • paypal won’t let you change them
• language only affects the login page
  • paypal then switch to user’s language
• process to approve:
  • get sandbox ID
  • submit test app to paypal
  • get live app ID
  • rebuild app (and don’t change it too much!)

Android In-App Billing

Richard Hyndman @geekyouup — Android Developer Advocate, Google

• available on 1.6 and above (since handled by Android Market app)
  • have to have opened the Market app
• two kinds of purchase:
  • Managed: once per account
  • can query market to find out if the purchase has been made
  • Unmanaged: unlimited per account
• get the Google Market Billing package from the ADK
• usage:
  • bind to service
  • check billing supported (requires network) — can then ask for upgrade
  • send billing request and put up billing page activity
  • set up billing receiver
  • purchase notify tells you something has changed, but you have to ask what…
  • quite a few back and forths…
• security
  • recommend that you put validation on server (urban airship?)
• use android.test.purchased as your SKU then it won’t get billed!
• use test accounts for testing…
• apparently the dungeons example has two or three classes you don’t actually need

Q&A

• subscriptions not in API yet…
• shared preferences can now be synced to the cloud
• to prevent copying to another phone, would have to check market when app launches
• currency tied to the google account, not to the IP location
• can only buy digital content/services

Paythru - Money on the Move

http://www.paythru.com/

• paythruMONEY — allows people to manage a card from their mobile
• can move money with a single-use QR code
• provides digital cash (user is logged in but retailer sees no identification)
• a few lines of code:
  • launch a browser to a specific URL
  • get the app to launch with a return URL when the paythru mobile web checkout has completed

Future of Mobile 08: Android and Gears for Mobile

Rich Miner — Google

Rich used to work for Orange before he co-founded Android.

Tom Hume also has good notes for Rich's talk.

• We have hit a tipping point with openness being a major catalyst
• Mobile phone operators tend to behave somewhat like lemmings…
  • (re: unlimited data plans)
  • Have probably been pushed a little faster & harder than they would have liked it (iPhone)
• Rich spoke about history of working at Orange, trying to launch SPV
  • Difficulties sorting out bugs — HTC didn’t have access to code, Microsoft said it would take months to fix…
  • App discovery was non-existent
• Fragmentation:
  • Compared number of mobile phones to number of cars, PCs, landline telephones…
  • Unsurprisingly, there are more mobile phones by several orders of magnitude
  • Even for Google, it was crazy to get apps signed on lots of operator networks
  • Google Maps for Mobile should be able to launch directly from your address book, but this is not possible in J2ME
  • It’s possible to access your address book, but only on a subset of phones that support the relevant API, and even then you have to implement your own search
• Showed slides from Android (the company) when being bought by Google:
  • Hardware costs going down but software costs staying the same
  • Existing Smartphone OSs aimed at enterprise & high-end, Android aimed slightly lower
• Android architecture — all available as open source: source.android.com
  • Linux kernel for hardware drivers
  • Libraries on top — SQLite, WebKit, OpenGL…
  • Android Runtime on top and to the side
  • allows access to all the data and services on the phone
  • App framework on top of both
  • Then actual provided apps on top of that
• Android marketplace is totally under control of developers
  • No human intervention between developer publishing and appearing on the marketplace

Mauricio Reyes recorded the Q&A if you want to see the video.

Q: What about the UI?

• Need to focus on consumer focussed user interface
• He didn’t quite answer the question, other than say it’s an important issue…

Q: What are plans to generate as much hype as iPhone?

• Not looking to replace iPhone — iPhone is already a good Google experience
• This is a 1.0 device from HTC and T-Mobile
• There are lots of others in the pipelines — there will be lots more arriving soon

Q: Once there are lots of different handsets, how will apps run on all handsets?

• cf. JavaME from Sun — there was no reference implementation, so each JVM behaves differently
• Instead Android has a single reference implementation so each device will have the same underlying software stack and apps will make the same calls
• Are working on a conformance test — for OEMs to run and carriers to use before accepting a test
• Google are also going to pick reference apps that challenge the platform (in a good way) and will highlight those as app tests

Q: WebKit and Gears

• At the moment, Gears is tied to browser app, not WebKit core
• This was a mistake and will be changed
• Will therefore be able to have a WebKit component in your app and still access Gears stuff

Q: Widgets for home screen not in current SDK

• Home screen is just an app
• Just ran out of time in development — have had lots of requests to add them
• Haven’t made a roadmap yet
• Since it’s just an app — can replace with something else — and some OEMs may do so

Google Developer Day 2008: A Deeper Look at App Engine -- Mano Marks

• goal of best practices: use less quota so pay less money and google have less load :-)
• free preview will always be free:
  • 500Mb storage
  • 2Gb bandwidth
• currently don’t allow you to pay for App Engine, but committed to charging for additional capacity by end of year
  • CPU: 10-12 cents/hr
  • Storage: 15-18 cents/hr
  • etc.
  • if double free preview, can expect about $40 / month
• currently support Python, but others will be coming
  • they know but they’re not telling
• looking to provide large upload/download support, but not sure how yet
  • current limit is 1Mb for file and response size
• currently no SLA
  • working with Cloud Status to monitor — http://www.cloudstatus.com/
• Google don’t put adverts on app engine apps
  • they will make more money on search :-)
  • They also don’t look at your data

storing data

• keys are limited to 500 bytes
• can’t change the ID or key_name
• transactional read & write with get() and put()

counters

• Bigtable doesn’t know size of tables — that would be O(N)
• Model.count() is a big transaction
• could create an entity that maintains the count
  • frequent updates can cause high contention
  • fundamental limitation of distributed systems
• instead, create sharded counters
  • randomly break counters into shards using a counter config to hold references to all shards
  • when want to count, ask counter config to add them all together
  • use get_or_insert() to fetch or create atomically

memcache

• when you add things to memcache, you define the staleness that you’re happy with
• use memcache to reduce storage and processing requirements

Q & A

• can use urlfetch to request data from your own servers

  • all app engine requests must complete in 10secs
  • and urlfetch must complete faster (4secs?)
  • you get an HTTP error that you can handle if the request times out

• Is there a profiling tool for app engine or Bigtable?

  • not aware of anything
  • difficult to see return on implementing memcache

• What logging is there?

  • there is a log; it logs requests and you can write to it

• SLA…?

  • the quota will be calculated using a moving average, not a total for the month
  • however, the aim is that if you get slashdotted you’ll stay up

• Parallel processes?

  • don’t allow threading
  • don’t allow direct file write access
  • have limited file read access
  • can’t access direct network sockets
  • however, there is a mapreduce implementation for app engine written by a Googler — http://code.google.com/p/httpmr/?

• email restrictions?

  • can send from any developer of app or from a logged in user
  • restricted to sending one per second

• when will Django 1.0 will be included?

  • some people have uploaded django 1.0 themselves
  • django 1.0 includes a C library, so this must be worked around at the moment

• three big languages internally in Google: Python, Java & C++

Google Developer Day 2008: Gears for mobile and the New Geolocation API -- Charles Wiles

• google maps for mobile took a long time to deliver on different platforms
  • 10+ platforms, 100s of builds
  • mobile updates every 6 months, cf. desktop every few weeks
• gears apps can be designed with UI talking only to local database
• background threads then bring in data as it becomes available
• how does Gears compare with HTML5?
  • short term — gears
  • Opera Mobile (WinMo) & Android Q4 2008
  • Expect S60 & BlackBerry in the future
  • long term — HTML5…
• geolocation to within 1km with wifi — enough for a lot of apps
  • two ways to call: one shot + event-based
  • typically takes about 5secs to get a fix

Demos

• Google Search with My Location — Andrei Popescu
  • plugs location and postcode into a mobile version of google local search
• fonefood — Marko Balabanovic
  • towards the beginning of the year looked at operator location
  • doesn’t work outside of home network
  • have to get multiple agreements to work around Europe
  • also looked at open source cell tower databases
  • patchy data
  • need to write a native app to get cell ID
  • now uses Gears Cell ID, works well for restaurant finding
  • shows Google static map with restaurant markers
• rummble — Clive Cox, CTO
  • can choose when to ask for permissions (call getPermission())
  • windows mobile:
  • always call db.close() before a page refresh
  • similarly, lastposition is only useful on a single page
  • google provide a whitelist/blacklist of user agents that support gears
• ITN Offline News — Ave Wrigley, Head of Technology Strategy & Innovation
  • ITN ON is the new media department of ITN
  • also have their own production team: showbiz, lifestyle, etc
  • generally produce content for other people, but are starting to go direct to consumer
  • use case: “flaky mode” — travelling to work with patchy coverage
  • watching videos as well as browsing news
  • also using geolocation for local news & weather
  • configuration interface — choose categories to browse offline
  • settings are on every page of the site
  • also forces that updates are grabbed wherever you are on the site
  • can find out through gears if the device is offline or online
  • don’t have to use XMLHttpRequest
  • not part of public API…
  • site grays out subscription options when offline
  • site warns you how many MB of video you are downloading
  • gears managed store is a bit of a black box — can’t tell how much you’ve downloaded
  • had to implement their own back-end AJAX call to find this out from the server side!
  • 80% of development time has been on platform — Pocket IE is effectively IE4…
  • they had to write specific libraries to cope with the crap browser
  • debugging is very difficult
  • code is in the managed store
  • need to make sure that you’re using the right version
  • developed techniques to get around that — ask offline

Q&A

• when Series 60 or BlackBerry?
  • sometime soon…
• will Google develop a gears translation layer for HTML5 browsers?
  • probably not — HTML5 is the standard, not gears
• will gears integrate with GWT?
  • probably soon

Google Developer Day 2008: Keynote -- Tom Stocky

• The browser is the platform, but it’s not powerful enough…
• Google was born on the web — it was a platform born of consensus
• Chrome aims to make the browser more resilient
  • one app is independent from the other
  • can automatically create app shortcuts like Fluid

• Gears for Mobile being developed by Charles Wiles in London

• app engine examples:

  • wordle.net — makes word clouds
  • buddypoke.com — one of the most popular open social apps
  • code.google.com/p/metasyntactic — now playing, iPhone app
  • one of top 20 free apps
  • pixverse — pixwall & pixchat, open social apps

Android — Mike Jennings

Supposedly the first live demo of a real Android device in the UK.

The big demo of accelerometer-enabled street view doesn’t work in the UK as there’s no data, and the rest of the demo looked a little flat. The UI was even a bit sluggish at times.

GWT — Sumit Chandel

• write really sophisticated web apps in the browser
• how do you handle GWT debugging? debug into Java code using “hosted mode browser”
  • runs your java code as bytecode, rather than generated javascript
  • can see stringified DOM contents of any widgets that you use
• demo: lombardi blueprint
• recently released GWT v1.5
  • java 5 language features
  • can use annotations to help optimisations
  • performance enhancements
• browser support (mid 2008):
  • Firefox 1.0, 1.5, 2.0 and 3.0
  • Internet Explorer 6 and 7
  • Safari 2 and 3
  • Opera 9.0

OpenSocial

• added Hive & Netlogs support
• Apache Shinding is an open source reference implementation of an OpenSocial container

OverTheAir: Google Gears for Mobile -- Andrei Popescu (Google)

Currently running on Windows Mobile as an ActiveX control. Just launched Picasa Web Album for Mobile with Gears. Works in IE 6 & 7, IE Mobile 5 & 6, Firefox 1 & 2 (3 coming soon). Firefox plugin works on Mac. Safari support being worked on. The Mobile version has the same API as the desktop version. Google have also announced that they are porting Gears to Android (no surprise there…) — this will be a Netscape standard plugin, so will probably come to WebKit at a similar time.

Gears is a browser plugin that provides additional APIs to the browser (local storage in a SQLite database, worker threads, local server for server data offline). See http://code.google.com/apis/gears/ for documentation and samples.

Mobile version also provides a Desktop API, which allows creating an icon shortcut to the app. Can provide multiple resolution icons.

Worker threads cannot access the DOM and the window object, so Gears also provides HttpRequest and Timer APIs for them.

Security: Gears will ask the user if a URL wants to store and access information locally. Thereafter that URL is allowed to use Gears to access data from the same origin (scheme, host, port).

BarCamp London: Introducing J2ME Polish

Here is the presentation I made at London BarCamp 3. It’s been a long while appearing on slideshare, but in that time the guys at J2ME Polish have released version 2.0, complete with all the features promised in the slideshow and more.

We use J2ME Polish at Kizoom and I’m really impressed with how easy it makes mobile development. You can’t forget about the fragmentation issue, but the issues you have to deal with are reduced to sizing images and choosing formats rather than writing code to switch between them.

| View | Upload your own

Future of Mobile: Android and the Open Handset Alliance (OHA)

by Dave Burke (Google)

Software development kit released on Monday: http://code.google.com/android/documentation.html

• Exposes Java for application developers -- SDK is based on Eclipse (works on Windows, Mac & Linux)

• Free phone stack provides at least 10% reduction of bill of materials for device
• Applications run as individual Linux processes with different user ids
• Complete two-way sync for user data (google gears?)
• Leveraging open APIs to expose application functionality

• Scalable to work with different device capabilities & input

• Component-based -- user can replace components
  • e.g.
• Overall architecture:

  • has freetype, media framework, SGL, OpenGL, LibWebCore (part of WebKit) libraries

  • Java runs in Dalvik Java VM
  • Four core apps: home, dialer, contacts, browser (can all be swapped out)
  • Can run MIDP applications in parallel
• Full source will be released with first handset
• Browser has visual history
• Can use Google Maps as a control in your own applications
• Example development process:
  • Emulator is real ARM emulator
  • Can create user interface declaratively using XML
  • Plugin picks up ids from UI XML and makes them available in java code completion
  • Can extend javascript in running browser component with native Java code
  • manifest file has permissions where can add security permissions
• Application Developer Challenge will distribute $10 million in awards to developers of applications for the platform
  • Developers retain all intellectual property and other rights to their applications while granting Google a license to evaluate and test the application for purposes of the Challenge as well as a license to display the application to promote the Android platform. More information will be provided when we make the Terms and Conditions available

• Android has simpler security system (for the user) than MIDP -- need details here

• Already works with real handsets but not available to public

• WebKit engine supports Netscape-style plugins

• Released under Apache license at the moment
• Designed for high-end platforms that include an MMU
• Own set of APIs providing full access to the device as opposed to MIDP
• Java is 1.5 so has generics etc.