Mobile Device Stats

I've been looking at various mobile statistics recently and thought I'd put up a quick post on my findings so far.

I started with some work at Kizoom, analysing the traffic we see on our UK Transport services. This then led to my session at BarCampLondon7, in which I presented the session counts for mobile browsers from a sample month (September) in 2007, 2008 and 2009. I've published these results in a DabbleDB database at http://mobilesessions.dabbledb.com/ for your slicing and dicing enjoyment.

I was inspired by Bryan Rieger's article on mobiforge, where he tells developers to "Expect and manage diversity" and links to some great graphs from the deviceatlas data explorer showing how screen sizes differ across device release dates. However, fantastic though deviceatlas is, it doesn't capture the actual usage of the devices on mobile sites -- it just counts one for each device, no matter whether it's a popular Nokia or an almost unused Sagem.

I've tried to rectify this lack of popularity data by publishing the Kizoom sample data. It's not perfect and each year isn't quite the same sample (we've released and withdrawn various services across the time range), but it's a start. I've even tried putting the same data into IBM's manyeyes to get some alternate visualisations.

Some of the trends that I've been seeing coming out of this data are not surprising: the iPhone punches way above its weight in terms of mobile traffic, and screen sizes are getting bigger as the years go on. One trend that is slightly unexpected is the growing popularity of BlackBerry devices over the last few years -- at least it's surprising to me quite how popular they've become (showing third in the brand popularity for 2009 behind Nokia and SonyEricsson, and ahead of Samsung and Apple).

There's been some other data published recently that seems to support the same trends. The September AdMob metrics have recently been released and this month they're highlighting the change in devices from 2007 to 2009 (a popular theme!). Their stats show that the iPhone and iPod Touch are number one and two in the UK -- probably due to in-app ads supplementing the web usage, especially when considering that the HTC Android phones (not nearly as popular a phone as the Nokia N95) also manage slots in the top 10.

Ian Homer of Bemoko pointed out the StatCounter Global Stats site, which has some very nice stats and pretty graphs to go with them. This site provides sliceable worldwide stats from 3 million websites (240 million hits in UK). Since the sites may not be mobile specific, I would expect mobile devices used to browse "the full web" would take precedence over smaller, more phone-focussed devices.

Here's an example: Top 9 Mobile Browsers in Europe 2008/09

As expected, the iPhone and iPod Touch (or "iTouch") gets the majority of the traffic, with Nokia coming a strong second. NetFront is the browser used by SonyEricsson smartphones (and others) so comes fairly high in the rankings (it seems that StatCounter may have changed how they categorise SonyEricsson browsers in August 2009, as NetFront swaps with SonyEricsson -- this is even more apparent in their Mobile OS stats for the same region).

Once more, BlackBerry devices are making a strong showing in recent months, a gain which shows even stronger when the stats are shown just for the UK.

BarCampLondon7: Alternative Input - Striking a Chord

Stephen Nicholas

3rd year project

• Stephen works with Dale Lane @ IBM on microbroker & GUI testing
• multitap & mini-qwerty not really suited for mobile input…
  • qwerty not really suited for desktop input either :-)
  • multitap gets up to 20 WPM
• braille keyboard has one key for each dot
• can get faster than qwerty keyboard input
  • qwerty 80 wpm
  • stenographers much faster: 200-300 WPM
• can possibly reduce RSI since not moving your arm quite so much
• drawbacks:
  • have to use non-obvious input schemes (no hunt and peck)
• microwriter, introduced in 1978
  • got speeds up to 40 WPM after 7 months practice
  • had great mnemonic method for encoding
  • recently been redeveloped as the CyKey
• twiddler
  • has 3 x 4 keys, might work well on mobile keyboard
• ChordTap for phones:
  • has three additional buttons on back of phone to distinguish between multitap letters

BarCampLondon7: Social Manipulation on the web and in person

Tim Nash

Social Engineering is still just as valid as ever…

• If asked for photo ID, people will give it up willingly
• given a photo ID with details, you can call up somewhere and say you’ve forgotten your ID
• often the front desk will give you a new ID…
• people are more likely to buy when there’s an official badge…
  • doesn’t really matter what the badge is
  • virtually no-one who clicks on the badge actually purchases
  • but buy rate will increase anyway
• paypal did an experiment about a year ago
  • to see if it made a difference to not use the word PayPal on their “paypal verified” badges
  • made no difference to sales conversions
  • click-through rate to paypal went down
• certain colours affect buying moods
  • brain doesn’t like bright colours
  • big red “buy now” button does not attract people — your eyes will avoid it
  • instead have a big grey “buy now” button with two red arrows on either side
  • eye tracking study shows people look for price and “buy now” button
  • sales rates go up when the price is just above the buy now button
• best place for comments on a blog:
  • new comment box just under post
  • other comments can be down the side
• use the word “reply” rather than “submit” comment
• scienceforseo blog
  • highlight a part of the text and the comment will appear alongside

BarCampLondon7: Project Lombok - the end of Java boilerplate?

Reinier Zwitserloot - @surial

Project Lombok is an extension to Java that allows you to write less boilerplate code by using annotations.

• examples:
  • @Data to provide automatic getters and setters for private fields, equality and hashCode, and a constructor for final fields
  • @Cleanup to tidy up inputstreams when block ends
  • @Synchronized to lock on private Object instance rather than this
  • @SneakyThrows() hides a checked exception from javac, but leaves it for JVM
  • (in JVM all exceptions are unchecked)
• integrates with Eclipse (e.g. getters/setters provided without you having to type)
• does mucking around through its jar, so can work easily with Ant & Maven etc
  • use com.sun...ProcessingEnvironment to
• Java Posse — podcast
  • inventor of annotations happy that lombok exists, but not so happy that private APIs being used…

BarCampLondon7: Supporting the masses

Tony McCrae

Experience from squadlist.co.uk — an online tool for organising rowing outings. Set up during a Guardian employee’s spare time for his own rowing club, but suddenly getting over 1000 users. He needed to support the users of the site without spending time on them.

• sort out forgotten logins
  • though have to be careful about security
  • generate nice passwords using pwgen
  • facebook isn’t permitted in some firms (so no facebook connect)
  • prompt end users to talk to someone else other than you!
• can your users be divided into groups?
  • form a relationship with one member in each group
• demo as documentation — install a full demo system with sample data and let them play
  • reset its data regularly (but make sure you tell people!)
  • Jira now has jQuery-based inline popup help
• invest in getting your domain unblocked
  • put SPF in your DNS
• even if you send from noreply@, check the volume going to it!
  • a spike may indicate something going wrong
• don’t tell everyone when changes happen
  • just senior users
• introduce new features to a smaller test group
  • enable beta group
• allow users to export their data automatically
• http://uservoice.com and http://getsatisfaction.com
  • with a forum on-site, people keep on requesting the same new features, even if you provide a roadmap
  • moving to these customer satisfaction sites means that existing feature requests no longer generate complaints!
  • instead, people vote up the existing requests
  • of course, if the requests never get monitored or picked up, you’ve still got disgruntled users
• really loud heavy users can be very powerful for good or bad
  • hire them! or at least make them happy
• gmail labs has “canned responses”
• open source core product to ensure continuity
• provide “paid for” additional capabilities

BarCampLondon7: Release London's Data -- Feedback from #londonsdata

Emer Coleman (assistant chief exec for London Borough of Barnet)

• has been doing research this year about what data has been collected in london boroughs
• big crisis in public finance after next election — what will happen?
• to have a meaningful conversation, you need to have information available…
• lack of trust at the moment (esp. for MP’s finances)
• GLA holds a lot of data and wants to publish
• already publish financial data
• hoping to put pressure on subsidiary bodies to release more
• invited developers to ask for what they wanted
  • 65 developers gathering information
  • TfL & transport info is really where they want to get to
• developers have gone as far as they can to push agencies
• “we’re just gonna do it”
• asked “do you want raw data, or with some contextual data?”
  • didn’t come to any firm view
• should we just release with the cabinet office?
• now an email list, and a google group set up
• will be running a competition
• aiming to release data store in January
• data avilable
  • air quality maps
  • number of people in social housing
  • school catchment areas
• message coming from developers — just put the data out there
• Daniel Shore (@LDN): what’s the attitude of TfL to scraping the journey planner?
• IanForrester: other authorities (e.g .Manchester) trying to do things too
• DIY Birmingham — scraped version of local authority site, but much more accessible
• Cloudmade: NaPTAN data being imported into OpenStreetMap
  • then allows it to be updated, corrected and extended
• Beth Noveck (Obama’s advisor on transparency) has written a book about wiki government

BarCampLondon7: Non-relational Databases

Simon Willison

Back channel notes on etherpad

• why?
  • scalability issues — have to do bizarre things to get to Flickr/Google size
  • some models don’t fit schemas
• Voldemort — used by LinkedIn
  • needs at least four servers to get started!
• CouchDB, MongoDB, etc
  • download and type make
  • MongoDB was much faster, tho’ CouchDB has improved
• whenever you hit a tag page on on flickr, you hit a search
  • if you hit “my photos, tagged X” you hit a relational database
• Programming the Semantic Web
  • by the guy who wrote Programming the Collective Intelligence — very good: all people who like X will like Y
• redis
  • key-value store, network accessible
  • ridiculously fast
  • doesn’t persist to disk — every 15 seconds it dumps the entire database to disk
  • can improve reliability by replicating
  • e.g. live stats services
  • can have a key-set, with add to set, set intersection
• Git
  • has shown that it can scale to the size of the linux kernel
  • so can scale to storing your desktop settings!
  • git is not just a RCS it’s a file system with revision control
  • there’s also git# and jGit
• jaiku migrated to app engine
  • including all the history
  • need to think of queries at design time, otherwise you’re stuck and have to do a big MapReduce to extract data

BarCampLondon7: iPhone Stats

Chris and Simon gave a good description of what it’s like to be popular in the Apple App Store.

• get an angle for the app
  • 0870 talked about in Guardian, twitter, TechCrunch, …
• 0870 download stats available on simon’s website
• marketing from Apple is not the be all and end all
  • and you have no control, so no warning when serious traffic hits…
• top 5/top 10 is the big hit
• when you put out an upgrade, 60-70% of new downloads will hit your app (and your feed)
  • again you have no control of when this happens
  • it’s usually at night (in the UK)
• hosting:
  • simon started on slicehost
  • went down after a couple of hours…
  • moved to rackspace cloud (engineX)
  • their cheapest server — has cost about $10 so far
• Kieran: tune apache so it can handle more connections at once since mobiles will talk for longer
• Ads:
  • good at the beginning but tailed off rapidly
  • Simon looking at premium ad suppliers
  • need 500K impressions/month
  • made $800 this month, but could make more on a premium network
• Q: Is it a point against you if you release your app on Cydia?
  • A: don’t think so, but then haven’t tried
• stats for paid:
  • if you’re lucky you’ll get 1000 a day
  • that will get you into top 25
  • a little more will get you top 10
  • if you’re focussing on a single territory, make it the US!
• iPhone apps are a great marketing tool
  • but probably won’t cover your costs…
  • Nigel saw a company that saw increase on their mobile web site when the iPhone app went live
• subscriptions are rolling:
  • app will warn you that you need to purchase a renewal
• Kieran: in-app purchasing is extremely successful — much more than subscriptions
• Even Nike couldn’t get app fast-tracked….!
• submitting & checking by Apple:
  • even new versions get treated as a new app
  • they only tell you one problem at a time
  • it’s about a 2-minute review — not a QA test
• AppViz — you need it for viewing your stats
  • nikf: there’s some equivalents that run on your iPhone (see markjnet’s review)
• 0.5% of users write a review, and less than that actually write a comment
  • paid apps get more reviews — people have invested
  • people don’t know how to do reviews unless they delete the app
  • appirater is an open source library that will prompt people to put in a review and take you to the right place on the app store
  • definitely put a feedback screen in the app
• putting apps cheaper at the beginning doesn’t seem to make much difference
  • there’s a bit chasm from free to paid, but then once you’ve made it paid there’s much less of a drop-off

BarCampLondon7: Energy Efficiency & Usage Monitoring

Nigel Crawley

• digital meter — LED flashes fast or slow depending on how much electricity you use
• can pick that up with an arduino with wifi and then make available as EEML
• EEML (eeml.org) — XML for electricity cost
  • can humm output eeml?
• can then input into Pachube
• can then do visualization like this:
• lilypad arduino — can sew into clothes
  • can include a vibra-ball
• can recognise individual devices by whole home electricity usage
  • fridge, kettle, toaster, electric oven
• Tom Raftery — greenmonk.net
  • devices using too much electricity — an organisation will offer to replace it and tell you the savings
  • Camden, New Jersey: government offers subsidies on lower energy appliances
• visualisations:
  • one for schools that showed a polar bear running out of iceberg
  • DisplayLink have done a blog post on energy visualisation

Gbenga Kogbe

• the UK will run out of energy by 2014… we must save
• comparing with your neighbours
• mancini project — plug by plug usage
• there was some effort in the Zigbee standard — all appliances would publish their usage to a standard hub
• energyhive provided reduced price meters
  • research that came out showed that by the end of the trial, loads of meters were in the drawer and not used
  • several 1000 homes included in trial
• putting information online and sharing it is much more effective than a little meter in the corner
• in some places, there are dynamic tariffs
  • would like to tell dishwasher to wash when it’s cheap
  • not in the UK…
  • energy providers buy at realtime, but sell at flat rate
• Dale Lane: energy costs vary between 2p and £3 a unit!!
  • it’s in their interest to get us to use it at the right times
• base electricity is provided by nuclear power
• peak is provided by hydro
• DynamicDemand.co.uk: figure out national supply by checking frequency
  • brownouts caused by frequency going too low
  • looking at making fridges turn themselves off when the frequency is lower
  • if all fridges did this, then peaks would be made less
  • see also caniturniton.com
• in California they have battery farms (since the 80s)
• solar panels are less efficient in the heat…
  • they get powered by light, not heat
• bike generators:
  • bikes available for free — have generators
  • when they are parked, they provide their power for the local buses