micro:bit music making

The year 6 Fleetville Code Club has kept going during all through lockdown – we spent a lot of time in Minecraft, which worked pretty well for virtual sessions, since we were all sharing the same virtual space.

This term we did something a bit different: the students have been making and programming MINI•MU musical gloves. This is a kit designed by Helen Leigh based on the MiMU Glove – in turn invented by the musician Imogen Heap and her team. You can get a glimpse of what the full MiMU Gloves can do in the following video:

The MINI•MU Gloves are considerably simpler than the ones in the video – instead of picking up all finger and hand movements, they use the movement sensors on a BBC micro:bit to detect basic hand motions, like how much you tilt your hand front to back and side to side. It’s still plenty of input to use for a musical instrument though!

We received the glove kits thanks to the OpenUK Digital Kids Camp and Huawei – who sent out 3000 kits and provided some cool e-zines which we used as a basis for our sessions. The students had to first cut the pattern from felt and sew their glove together, then connect the micro:bit, sound board and battery pack – before programming the micro:bit to make their own sounds.

This was a harder project to get to work in an all-remote environment, and I think some of the students found it hard to stay engaged without being able to share their creations in real life. Even so, I was really impressed with what they managed to build. Here’s my favourite wishing one of the student’s grandparents a happy birthday!

The @FleetvilleJnr year 6 @CodeClub have been making and programming MINI•MU Gloves this term – here's my favourite recording for the #MicrobitVirtualConcert with @jrflorentine & @TinkeringStudio

Thanks to @openuk_uk for providing the kits designed by the amazing @helenleigh! pic.twitter.com/d7nJ7z32Uc

— Adam Cohen-Rose (@adamcohenrose) May 21, 2021

Taking it further…

The MINI•MU gloves make it easy to make a sound as they have a built-in speaker. However, the micro:bit can only make very simple beeps by itself.

I spent a bit of time trying to get closer to the full MiMU glove experience by sending the movement of the glove to something else that could make some richer sounds.

Scratch

One option is to connect the micro:bit to Scratch using its bluetooth connection. The Scratch site has very clear instructions on how to get the micro:bit connected – you need to install the Scratch Link software on macOS or Windows, or else use the Scratch app for ChromeOS or Android tablets; then install a special program on the micro:bit itself; and finally add the micro:bit extensions to Scratch and identify your micro:bit by the name it displays.

There’s a few tutorial programs to get you started on the Scratch site, and there’s a range of examples on the microbit site as well – including a theremin to make spooky sci-fi noises.

I’ve made another example that lets you make noises as you move either the mouse or the micro:bit around. You can have a go below:

Bluetooth MIDI

Another way to get better sound from the MINI•MU glove is to connect the micro:bit to a tablet or computer using MIDI over Bluetooth. MIDI is a music communication system that’s even older than the micro:bit’s BBC precursor from 1981, the venerable BBC Micro. You can use MIDI to describe starting and stopping a particular note, how hard the note is played (its velocity) and all sorts of other variations – and the system is supported by most music making programs, including things like GarageBand.

To get MIDI from your micro:bit to your computer or tablet is a little complex but once it’s working, you can make all kinds of sounds! Here’s some tips and tricks to get you going:

Build a project with the Bluetooth MIDI extension

To send Bluetooth MIDI from your micro:bit, you’ll need to add the Bluetooth MIDI extension:

1. In MakeCode, click Advanced at the bottom of the blocks
2. Scroll down to the bottom to see Extensions
3. Click that and type “midi” into the search bar
4. Make sure you click on the bluetooth-midi extension, not the plain midi one
5. You’ll now have some extra light blue blocks in the Midi category

Here’s a ready-made project you can start with – Pentatonic MIDI controller

Tilt your micro:bit left and right to choose a note, then press the A button to make the micro:bit send the current note. There’s a secret feature to use the light levels picked up from the LEDs – can you figure out how to use it?

Make your micro:bit easier to connect

micro:bit MakeCode projects have a secret option that makes it easier to connect the micro:bit to something over bluetooth. It would probably make things harder if you had lots of devices in a classroom, but if you’re just wanting to connect a single micro:bit to your computer or tablet then this will make your life much easier!

1. Go to project settings by clicking on the gearwheel at the top right of the page
2. Turn on “No Pairing Required: Anyone can connect via Bluetooth”
3. Click Save
4. Re-download your project to the micro:bit

The Pentatonic MIDI Controller project above already has this option set for you!

micro:bit to iPad

This is fairly simple – in GarageBand for the iPad, go to Settings (the gearwheel icon), click Advanced, click Bluetooth MIDI Devices, tap the micro:bit then tap the Connect switch on. If the micro:bit is marked as offline, click Edit and Forget the device.

Now you can use your micro:bit to play and record tunes in GarageBand – and choose any instrument you want!

micro:bit to Mac

macOS won’t connect to the micro:bit by itself and needs a little bit of prompting. Download and run the Bluetility app to see the micro:bit, then click the last service and the last characteristic (they should start “E95D93AF-” and “E95DB84C-” respectively). You should then see a Subscribe button to click in the Detail pane – and this should then make the micro:bit appear in your Bluetooth system preferences.

Now that the micro:bit is connected over Bluetooth, you need to tell the macOS MIDI system that it needs to listen to it. Open the Audio MIDI Setup app (it lives in /Applications/Utilities – or just use Spotlight Search). Make sure the MIDI Studio window is showing (if not, go to the Window menu and choose Show MIDI Studio). Then click the Bluetooth symbol in the MIDI Studio window’s toolbar, which should open the Bluetooth Configuration window and list your connected micro:bit. Click the Connect button next to your micro:bit and you’re finally ready to play!

Open GarageBand, create a new project, and add a Software Instrument. GarageBand should pick up the connected micro:bit immediately and play the notes that you send. You can then choose different sounds from the Library to make your micro:bit play whatever instruments you want.

micro:bit to Windows 10

I haven’t tried this, but there’s a very clear video by BEATNVISION that suggests using the MIDIberry app to receive the Bluetooth MIDI from your micro:bit and play sound.

Use the micro:bit connectors

As well as the movement sensors, the micro:bit also has edge connectors that you can use to attach all sorts of other technology. The MINI•MU glove uses these to connect the speaker, but if you’re going to use bluetooth to output the sound, then you can use the connectors to get more input.

Here’s the micro:bit wizard David Whale with his micro:bit guitar:

And if you really want to see how far you can go, how about using muscle movements to control your music? Here’s a video on how to pick up electrical changes from your muscles using a micro:bit:

Keep making music!

The Web Is... Day Two

Took me a little while to prepare my notes for the second day of The Web Is… conference, mainly due to an ‘orrible cold that developed over the last week. I hope I haven’t given it to the lovely people I spoke to over the last few days!

Thank you once again to Craig and Amie Lockwood and all their helpers for a fantastic conference. I know they had a lot of trouble getting the conference organised, but the end result was very definitely worth it.

Craig — please keep organising more conferences! You have a rare skill in finding and curating talks that connect at many different levels —- and that challenge us to do better and improve the world.

Here are my notes for day two, complete with the additional Creative Morning session.

Creative Mornings: Crossover

• creative mornings: 99 cities around the world
• meeting once a month with a common theme across all
• this month, the theme is Crossover

David Hieatt, Hiut Denim @davidhieatt

• co-founded Howies
• runs the Do Lectures
• entrepreneurs fall in love with things so they don’t quit
• have to choose something that you’re interested in
  • it will get tough and test you
• vision: what have I seen that others haven’t
• intersection of skill, interest & vision
  • where you’re most alive
• fight a battle you can win
• denim factory in wales used to be biggest jeans factory in UK
• employed 400 people, made 35K pairs of jeans a week for decades
• tried to be cheapest — not the right battle
• in UK have to use our creativity
• dreams should not be logical or sensible
  • should scare you!
• currently employing 14 people — quite a way to go
• but everyone in the town has decades of experience making jeans
• there are people in the factory who have 50,000 hours of experience
• the elite makers club
• but quality isn’t enough
• need to find ideas to act as multipliers
• hiut denim: an ideas company that makes jeans
• Richard Seymour: the more you operate in the future, the less competition you will have
• lots of small experiments
• small company — move fast
• jeans have history tags on them
  • 6 photos of them being made
  • plus you can upload new photos for their history
  • when the jeans get handed down, you can see their history
• 80% of environmental impact is the washing, not the making
  • hence no wash club
  • people doing crazy things to not wash for 6 months or more
• people want to wear used jeans
  • so sent out jeans to people to break in
  • they would get their deposit back when returned
  • then receive 20% of resulting sale
• facetime store
  • will create a custom fit for you from a facetime call
• instagram is going to change stores
• your best project is the one that’s in front of you now

The Web Is… Like Water

Scott Jenson @scottjenson http://google.github.io/physical-web/

• standardising pull to refresh, just like the steering wheel…?
• but steering wheels started as tillers with rear wheel steering
• User Unfriendly, Joseph J Corn
  • people preferred the tiller at the time
  • needed irreversible steering to prevent too much feedback from potholes
• browsers have a DOS prompt on the top…
  • the browser needs a browser
• the web = Loki — proud, vs iPhone = the Hulk
  • “I am a God, you dull creature. I will not be bullied”
  • …body slam…
• the web is currently following native, not driving the future
• we have app myopia
• “thin crust of effort” around apps
  • people will only install them if they’re important
• apps are our technology tiller…
• Apple iBeacons are per-app at the moment
  • go into a mall — install one app for each store…
  • then delete them all afterwards as they’re all crap
• the web is good at the long tail
• break down the internet of things into several layers
  • coordination
    • devices talking to each other
    • toast automatically ready when you come down for breakfast
    • cool but will take a while
  • control
    • where we are right now
  • discovery
    • commonly ignored — usually “download our app”
• think small — just a bit of information for each thing
• Bluetooth Low Energy beacons just broadcasting a URL
  • like QR codes but without having to get so close

• URL with information
  • bus stop
• URL that sends info back to the device
  • vending machine
  • GSM modems are really cheap
  • Sigfox offers $1/year plans
• URL with javascript talking BLE direct to object
  • object can make use of all the capabilities of the phone
  • so much cheaper per device
  • Chrome already has Bluetooth library but behind Chrome apps
  • but creating a proposal with Mozilla to put into web sandbox

• two kinds of ideas:
  • truck ideas — products
  • road ideas — infrastructure
• Malcolm McLean — invented container shipping
  • created lots of patents then gave them away
  • realised he would make more by having a smaller percentage of larger market

• physical web beacon
  • currently no security etc — just for testing
  • configure it by pressing the button
• the Google Chrome team is staffed by boy scouts
  • just want more URLs in the world

The Web Is… Still Young

Benjamin Hollway @benjaminhollway

Slides available at http://nothingrandom.com/says/thewebis/young

• how can we support young coders under 18?
• cheap/free tickets for conferences
• opportunities to speak
  • unique perspective
• alternative after party venues
  • accessible to people who can’t drink or are too young
• help introduce kids to computing
  • join or set up a Code Club
• mentoring
  • work experience
  • paid internships — 2-3 weeks during summer holiday
  • deeper insight into how the industry works
• future
  • inclusivity
  • < 5% use IE, but at least that or more have disabilities

The Web Is… Turning Difficulties Into Opportunities

Robin Christopherson, @USA2DAY

• AbilityNet: leading experts in technology & disability
• also do assessments: one of first was Prof. Stephen Hawking
• technology can really help people with disabilities
• light detector
  • helps to see if lights/indicators are on or off
  • especially for enabling oven light (they don’t make speaking ovens)
• talking goggles
  • reads any text from camera
  • also recognises some images
• word lens
  • translates and can then read out using voiceover
• TABs — temporarily able bodied
  • it’s likely everyone will have some kind of some kind of disability eventually in their lives
• also mobile use cases are often very similar to accessibility cases
• a blind date with an iPhone
  • woman prepares all her responses before the date…
• google glass captioned audio
  • speak into Android phone, see captioned audio on glass screen
• FaceXpress
  • identifies emotions from faces
• live facial recognition
  • identify people coming to your door if you’re bed-bound
• directions in maps apps
  • audio directions are very powerful
• Moto Hint
  • Google Now as an earbud
• IBM Watson now has a developer kit
• self-driving cars
  • 30% of city driving is looking for a space
  • eliminating human error accidents
  • need to have speech output
  • otherwise can’t tell if arrived or just stopped at a traffic light!
• direct brain interface
  • completely paralysed woman able to move a robot arm
• technology4good lifetime award given to Prof. Hawking
  • acceptance speech worth listening to
• 73% of people with a vision impairment are out of work
  • there’s no longer any reason why they can’t work

The Web Is… Too Slow

Andy Davies @AndyDavies

• shopdirect (run littlewoods brand):
  • on average, each purchase is across 2.4 devices
• Tammy Everts: only 12% of top 100 US retail sites render feature content under 3 seconds
• we perceive delays around the 1s mark
  • 100ms appears instantaneous
  • 200-300ms fast enough
  • 1s start to feel it
  • > 1s we start to task switch
• the web is for everybody …but only if we make it that way
• Chris Zachariah: made youtube pages faster
  • but time to view video went down
  • turns out they were reaching new audiences
  • that hadn’t been able to use the site at all before
• Guardian’s new design:
  • speed was a feature in the press release
• Walmart have released conversion rate vs load time
  • all conversions concentrated in first 4 secs of load time
• Staples increased conversions by 10%
  • cut 1s from median home page
  • 6s from 98th percentile
• we’re more tolerant of slow experiences in some cases
  • when we’re more committed, e.g. further into conversion funnel
  • when we’re used to it
• kayak study: we don’t trust comparison sites that are too quick…
• Matias Duarte: “design is finding solutions within constraints .. if no constraints, it’s not design but art”
• need to set a performance design constraint
  • e.g. an event that matters to the user happens within a certain time under certain network conditions
• doesn’t mean anything to measure page sizes or number of objects
• Etsy: have lots of feature flags
  • all staff see a performance bar at the top of the page
  • indicates if the page has met requirements
• PerfBar
  • plugins for different browsers and sites
• WebPageTest
  • can run tests & set a budget based on competitor sites
• speedcurve
  • used by the guardian
  • have a performance chart outside the financial director’s office
  • performance budget lines coming in November
• headline network speed doesn’t include latency
  • often matters more than bandwidth
• network connection closer to a bucket brigade than a pipe
• TCP has a slow start algorithm that directly affects initial page latency
  • more modern operating systems increase the initial congestion window
  • this increases the number of segments that can be sent without an acknowledgement
• rendering:
  • CSS Object Model blocks Javascript execution
  • Javascript blocks DOM construction
  • complex CSS can really slow the initial rendering
• dealing with fonts
  • Ilya Gregorik suggested new font-timeout & font-desirability properties
  • would allow us to choose to render the page without the font
• HTTP/2 has server push
  • server can push additional files that it knows the browser will want
  • if the browser has a cached version, it can reject the push

Slides will be available at http://www.slideshare.net/andydavies (eventually)

The Web Is… Our Responsibility

Sally Jenkinson @sjenkinson http://recordssoundthesame.com

The Web Is Our Responsibility from Sally Jenkinson

• concept keyboards provided touch-based input to BBC Micros back in 1984
• symbiotic relationship between technology and experiences
• remember it’s as much about people as it is about development
• all the roles need to work together
  • e.g. performance cuts through all layers
• responsive design mostly seen to be to do with screen widths
• but there’s much more (especially in CSS level 4 draft)
  • whether the device supports hover affordances
  • the current ambient light level
• invisible requirements
  • the technical choices that we make underpinning everthing
  • e.g http://offlinefirst.org/
• Open Data Institute guest book
  • iPad with keyboard
  • can sign in with touch or typing
  • or can add an existing RFID card (e.g. Oyster or bank card) to check-in in future
• GE Healthcare adventure series
  • MRI scanner dressed up as spaceship
  • children can imagine the spaceship rumbling through space
  • much less scary
• be aware of the choices we make
• try to think beyond ourselves
• you are responsible for things you build
• if someone presents you with something that you don’t agree with, then question it
  • even if it’s not officially your responsibility
• try and educate people as to why things should be done differently
• if you are in a big project that you disagree with, try and make little changes

The Web Is… Read/Write

Owen Gregory, editor @FullCreamMilk

• acts as a conduit between writer and reader
• the web has a closer relationship between writers and readers
• publishers & editors need to be advocates in their community
• jason santamaria: where is the cultural and aesthetic discourse on web design
  • writing is mostly about the new techniques and looking forward
  • where does a common language for web design discourse start?
• jakob nielsen: we should accept that the web is too fast moving for big picture description
• book: nicely said: writing for the web with style and purpose
  • nicole fenton & kate kiefer lee
• Sir Thomas Browne: The Garden of Cyrus
  • drawing parallels between lots of different fields
• W G Sebald: The Rings of Saturn
  • new style of writing, referring to the author
  • tracking Thomas Browne’s work
• deploying a text can be a continuous process — Mandy Brown
  • comments, talks, reframing, etc.

The Web Is… Progress

Mr Bingo, @mr_bingo

instagram: mr_bingstagram

• apparently the web is blue…
• D&AD brief: put the fun back in to the web
• hand-drawn aesthetic — much easier in flash…
• 2010 selling prints of drawings for £5 each for Haiti earthquake
• internet commenting is like toilet graffiti
• worried that if I die, when my life flashes before my eyes, all I will see is square pictures of food
• weird stock photography… (just watch the talk!)
• opened a service for sending people insults on postcards
  • HATE Mail — and now a book

The Web Is… Progress

Brad Frost @brad_frost

• web enabled linking to anything we want (was a radical change…)
• open by default
• the most sharing community in the world
  • e.g. css-tricks license
  • “I want the web to get better and being all Johnny Protective over everything doesn’t get us there”
• pittsburgh food bank
  • designed and built a new web site for them
  • did the entire process in the open
  • http://foodbank.bradfrostweb.com/timeline/
• why share?
  • because we can — it’s cheap
  • but it can be really uncomfortable
  • other people can contribute
• trying to solve everyone’s problem is in some cases easier to solve just your own
  • an idea can snowball into a plugin just a few hours later
• our first drafts can have their own value
  • in some cases they can have more value than the finished work
• the web is for everyone — but that means all the violence, hate and anger as well…
• guardian, bbc and gov.uk designing in the open
• other companies and even other countries can use this as a blueprint for their own work
• each of us is individual and has unique different experiences
• knowledge is an iterative and cumulative process
• depending on sharing and combining our different perspectives
• designopen.org
• even if lots of people contribute similar things, they still contribute
  • e.g. 162,000 youtube videos on how to re-wire a plug
• your post might not be the most original, but it might be the one that tips over the scales
• it has never been easier in the history of the world to contribute to the knowledge of the world

NSConference 5: Day Two

Thriving in an App Store World

Michael Jurewitz @jury

Jury used to be an Apple Developer Evangelist. He is now Director of Product Development at Black Pixel.

work well with apple

• need to be looking forward - focusing on future (tech/features/hardware)
• stay on the radar of apple contacts
• Apple looks at devrel as “animal husbandry” :-)
  • Apple wants to sell devices & make customers happy
  • so apps should feed into that
• Apple has laser focus on future and simplicity
  • expecting & embracing change
  • they force themselves to keep going forward
  • won’t look at apps that don’t fit in to that
  • must stay current
• no secret agenda, but often hard choices
• can get featured by taking advantage of new OS features
  • but it’s a time limited offer…
• example:
  • Eventbrite getting featured on Passbook feature page increased new user sign up by 664%…
  • got in right at the beginning – only 15 apps on the feature page
• “if users aren’t upgrading their OS, they probably aren’t buying your software either” - Wil Shipley
• Gatekeeper vs App Store: e.g. Kaleidoscope
  • if you buy Kaleidoscope on Mac App Store then you can also download direct
  • app will notice that it’s already been purchased
  • will check receipt and unlock automatically
  • useful for sending customers beta builds when testing fixes
• file bugs
  • also for requesting to open APIs that are currently private
• localisation:
  • Apple sales in China increased 400% last year
  • Germany has a strong software market, even though it’s limited in size
• accessibility creates loyalty

properly value your work

• Jury did some research over the past weeks…
• top paid apps have much lower mean & median prices than top grossing apps
  • mean: paid $12.46 vs grossing $49.13
  • median: paid $6.99 vs grossing $29.99
• two separate markets going on here
• top grossing has:
  • 200% more Finance apps in top grossing at the moment
    • because it’s tax season at the moment in the US
  • Utilities drop by 25%
  • 22% more Games
  • 50% more Graphics
  • Social drops in half (and would be nothing without Tweetbot)
  • No Weather apps
  • 3-5 Business apps vs none in Top Paid
• four free apps in Top Grossing
  • in-app purchases
  • freemium can work
  • talk to Kevin Hoctor who did in-app purchases in Moneywell Express
• cheap apps get downloads, but higher priced apps pay the bills
  • which would you rather have…?

economics 101:

• price elasticity of demand
• how does adjusting the price of a product affect the number that you sell?
• elastic (e.g. social apps):
  • there’s a common price that people expect
  • if you increase the price beyond that, then demand massively decreases
• inelastic (e.g. drugs… or photoshop…):
  • people will pay pretty much whatever you ask
  • keep increasing the price and maximise revenue
• demand curves can move due to marketing campaigns…
• if you double the price, and you lose less than 50% of your customers, you’ve just made money
  • and fewer users means less support costs
  • also increases perceived value
• don’t be a commodity — charge what your software is worth

price your app intelligently

• market segment:
  • lots of basic research
  • category prices
  • start research before starting developing
• research competitors
  • do you have a big enough differentiator?
• make a guess
• then try an experiment
• example: Kaleidoscope 2
  • developer tool segment: avg price $30.11
  • other apps $70-100
  • app is useful but not crucial
  • lots of alternatives
  • guessed $34.99 intro and $69.99 ongoing
  • 
  • got various peak sales that affected average
    • great for recouping costs, but not for calculating ongoing revenue
  • evaluation:
    • elasticity = % change price / % change in quantity
    • with a couple of price changes, you can work out the ratio
    • then apply to price changes to predict quantities and therefore revenue
    • (real world demand curves aren’t linear, so elasticities aren’t actually constant)

Talking to Hardware

Alasdair Allan, Babilim Light Industries @aallan

• Apple’s External Accessory Framework is missing most of the useful stuff
  • rest is protected by Made for iPhone program
  • which is protected by massive ranks of lawyers
  • because Apple want to protect their platform

crazy stuff

• jailbreaking
  • average time between a jailbreak release and Apple shutting the hole is about 7 days…
  • can’t release an app to the store
• MIDI
  • iPad camera connection kit supports MIDI
  • https://github.com/bjepson/iPad-MIDI-Simple-Demo
  • can subvert MIDI to support serial, but it’s horrible
• simulate capacitance touch using a piece of foil stuck to the screen
  • it looks nuts when you experiment
  • but http://misfitwearables.com does exactly that to sync its data
• PeerTalk: using the USB sync cable
  • uses TCP sockets
  • same protocol as iTunes and Xcode

less crazy stuff

• wifi is possible, but getting setting up is a pain
  • and there’s lots of support issues for different network situations…
• acoustic coupling via the headphone jack
  • Square does this for a card reader
  • can even use the audio to provide 7.4mW of power (see Hijack board)
• Redpark cable
  • dock connector to RS-232
  • comes with an SDK
  • but won’t let you put apps in the store — have to approve the app and the hardware
    • you can approach Redpark and ask them to request approval
    • a lot less expensive than going through MFi program
• XBee & Zigbee (802.15.4)
  • mesh networking for low data rates
  • dock connector to XBee adapter soon available from redpark
• bluetooth 4 (low energy)
  • can run for months with bluetooth active powered by a coin cell
  • introduced with the iPhone 4S
  • lots of boards available with Android & iOS SDKs
  • e.g. red bear labs
  • use CoreBluetooth plus board’s SDK
  • easy integration:
    • Alasdair did a live demo in just a few minutes
    • connected to arduino over Bluetooth 4 from iPhone and toggled an LED

TouchDB

Matias Piipari @ms2

https://github.com/couchbase/couchbase-lite-ios

• CouchDB has been renamed as CouchBase Lite
• TouchDB is document db with CouchDB-like API
  • but uses SQLite under the hood
• sync & share with CouchDB (or TouchDB)
• concurrency controlled like git
  • when you’re saving you have to be up to date first
• lightweight
  • < 500Kb in app binary
  • 0.1s startup
• Document <=> Model, with versions
  • can also contain attachments
• use Views to define queries by property
• once you’ve configured the replication it will keep going
  • don’t need to worry about network availability
  • do need to think about conflicts
  • do need to think about concurrency with data changes (get notifications on changes)
• can create push and pull separately and to/from different destinations
  • e.g. combine pull from bundled with pull from remote
• sync handles https with basic auth or OAuth
• can combine db from iCloud/Dropbox with TouchDB

Step away from the screen

Nathan Error, Empirical Development @neror

nathan@empiricaldevelopment.com

• your body is a tool too: improve your coding by improving your body’s effectiveness
  • exercise, diet & sleep…
• scientometrics
  • measuring rate of change in science
  • rate of output increases by 7% each year
  • output doubles every 10-15 years
  • so whatever we think now will probably change several times over the next 10-15 years
• maybe start looking at meat more as a side dish rather than a main
• recent research has linked high intensity aerobic exercise to increased brain performance
• missing one hour of sleep for a week is equivalent of a blood alcohol level of 0.2%

Subscription pricing

Manton Reece @manton on ADN

manton@manton.org

• in 1999 working on a Mac app that cost $199
  • actually considered pretty cheap for the time
• benefits of subscriptions
  • happy customers:
    • unhappy customers can cancel at any time
  • automatic paid upgrades
    • everyone is on the latest version
    • paying for the service
• to justify subscription, app and service need to be one
• examples:
  • adobe: switching to creative cloud monthly subscription
  • microsoft: office is now $9.99/month (or $99.99/year)
  • billings pro:
    • free for 1 invoice/month
    • 5 invoices/month = $10
• focus on the consistent predictable part of the graph rather than the spikes
  • even if you don’t make more sales, the revenue is consistent
• billing periods
  • payment percentages affect revenues: charging less often means less percentages to payment provider
  • Manton found that 57% of customers preferred yearly billing
• hosting costs
  • use Amazon reserved instances if you’re committing to a year (saves money)
• Stripe is leaps and bounds better than paypal…
  • but only available in US and Canada
  • beta coming to the UK this week!
• Apple in-app purchase types:
  • non-renewing subscriptions: cancel is the default — will probably lose a lot of people
  • auto-renewable subscriptions: better, but more restrictions on review — including privacy policy & description
    • Apple are very cautious about letting non-magazine apps do this

The “Simple And Intuitive” Fallacy

Why we need standards for complex UX, too

Joerg Schweider @cooliopenguin

iPeng

• iPhones & iPads are not just accessory devices — they are becoming the main device in a lot of cases
• apps need to be feature complete
• if you simplify and cut out features then a lot of users will be left out in the cold
• what makes a more intuitive UI?
  • can’t always find out by getting people to compare UIs
  • they will rate familiar schemes higher

Being Naive

Rob Rhyne @capttaco http://martiancraft.com/

• “just build something so I can show it to the client”
• users don’t care about engineering
• iterate and test
• at martiancraft, within three weeks of a new project you’re going to see something
  • it won’t be finished, but you can play with it
• brent simmons: anatomy of a feature
  • the feature is the smallest part
  • it’s all the edge cases and polish that take the time
• the naive implementation
  • demonstrates the feature
  • most obvious solution
  • takes the least amount of time to develop
• example: histogram of live video
  • can use Accelerate framework to get histogram data from an image buffer
  • what about drawing the graph?
    • 3rd party charting lib: not obvious; little work; may demo but might not animate
    • OpenGL vertex buffer: not obvious; lots of work; will demo
    • CoreAnimation: obvious (when experienced!); little work; will demo
  • used CAShapeLayer
    • had used previously in Minds of Modern Mathematics
    • 14,000 pixel wide scroll view on an original iPad
    • can set up style once and draw separately
    • path was animatable property
• http://giveabrief.com
  • codeless prototypes

Working on Sketch

Pieter Omvlee, Bohemian Coding

• Sketch: vector art app
• have to persuade users to try us rather than Adobe
• don’t want a public feature list with voting
  • sets unrealistic expectations
  • there will always be business goals or technical issues that mean that highly requested features remain unfixed at the top
• listen to your customers, but only to a certain degree
  • sketch run a tenderapp discussion board
• started giving away beta versions of app
  • got lots of testers & feedback
  • keep the fans happy
  • they’re very vocal and will do marketing on your behalf
• don’t leave refactoring until the next big update
  • you’ll want to focus on new visible features to justify the upgrade price
  • customers don’t care about engineering
• be practical with your time
  • don’t spend time on rewriting git history
  • focus on pricing, attracting the right customers, etc
• put in crash reporting early
• videos are excellent promotion & support
  • they take time and lots of takes
• try and keep in contact with your customers
  • Apple gives you no way of getting in touch with App Store customers
  • but you could ask for details within the app
  • if no newsletter, then Change Logs are your only communication medium
  • “bug fixes” is a waste of an opportunity
• try and contact bad reviewers

Introducing CoreValues

Scott Morrison (Chief Cook & Bottle Washer), Indev Software @smorr

smorr@indev.ca

• Indie developer, less Independent, more Individual
• but not just single person
• instead individuality & personal
  • personal investment, principles, impact, payoff & risk
• personal & professional roles are mixed
• indie company has the heart of the indie developer
• values are important and define you
  • but sometimes not thought through
  • can lead to inefficient decisions
• Edward de Bono: if you want an out of the box solution, get out of the box first
  • PO statement — an unconventional (silly) idea used to generate ideas
• introducing CoreValues: an objective-c framework for describing and defining personal and professional values…
• interfaces:
  • personal ≠ private
  • professional ≠ public
• (there was much more in this vein: thought-provoking metaphors, but I didn’t write them down)

Slightly Unsupported — Finder Code Injection

Steve Flack, Bromium UK

• wanted to add icon badges & contextual menus
  • not based on file type
  • animated
• used class-dump, x86 disassembler plus lots of trial & error
• inject code with mach_inject
• Finder has four methods to swizzle for icon badging
  • one for each view (desktop is fourth)
  • talk gave code specifics for each
• only two methods to swizzle for contextual menus
  • desktop & grid view share
  • again, talk gave specifics

The Rise and Fall of a Mobile Startup

Emily Toop @fluffyemily

emily@emilytoop.com

• Tiny Ears 2011-2012
  • app to teach reading to 4-6 year-old children using speech recognition
• Emily and her partner Ian both got startups accepted to Start-Up Chile
• had to get speech recognition
  • Google Voice API was best accuracy by far
  • PocketSphinx with OpenEars was only option with no network
  • but all were bad for children
• can improve recognition with a better model
  • 20 hours to customise a model
  • no existing models for children
  • 20,000 hours to create a new model!
• Startup Weekend
  • pitch your product
  • then recruit audience to work on it for 48hrs
  • make an MVP
  • really useful
• talk about your product
  • to everyone
  • until you bore everyone
  • even yourself…
• used AVAnimator to play movies with alpha channels
• …but speech recognition model would cost $100,000 and take 18 months
• then animators didn’t want to continue without speech recognition
• make your partnerships sound!
• don’t go alone
  • guides recommend that you have a hacker and a hustler
• mothballed Tiny Ears
• joined another startup to learn how startups work
• getting more involved in education
  • joined Code Club
  • applying to be a reading assistant
• approached by dreamthinkspeak to help with their “large-scale, site-responsive theatre production inspired by Leonardo Da Vinci, The Book of Revelations and the world of Mechatronics”
  • go see them while they’re still in London (until March 30th at Somerset House)
• nothing you learn is ever wasted

Google I/O 2011

Some brief notes on the Google I/O 2011 keynote this afternoon… It started kind of dull with Android stats that were pretty good but still seemed to be catching up on Apple, then gradually got better and better.

Google really does want Android to be “one OS everywhere” (I guess running “one web” webapps…) and the UI is improving. I look forward to the media content coming across to countries other than the USA, though the lack of any mention of music or movie labels leaves me in some doubt as to when this will happen. The presentations neatly glossed over the time involved in syncing music and movies between devices and the online service — even on a home wired network the data would take a long time to upload. And there was no mention of syncing between devices on the same LAN like Dropbox either.

The big news for me, however, was the accessories and android@home. By making it so cheap to add hardware integration with Android devices, there could be a real explosion of new products and the home automation market could finally break out of eye-wateringly expensive or home hacker territory. I’m really looking forward to more detail on android@home, but there’s nothing released yet…

android updates

• widgets can now be stretched
  • scrollable -> resizeable (just change XML)
• android can now be a USB host
  • keyboards, mice, game controllers
• google tv market open for devs this summer
  • will be based on android 3.1
• icecreamsandwich coming Q4
  • advanced UI
  • APIs to help scale UI
  • action bar able to reconfigure itself for available space

media

• movies and books now available through android market
  • can watch on web or on android device (e.g. android TV…)
  • seems to be US only (like Music)
• google music
  • cache music recently played
  • can “pin” music just like on movies to make available offline
  • downloads in background
  • 20,000 songs can be added to library
  • free in beta…
  • new music app works with android 2.2 and above

hardware

• new devices will receive latest updates for 18 months after first launch
  • (if the hardware allows)
  • guaranteed to come through reasonably quickly
• android open accessory
  • plug in an accessory
  • phone gets taken to app (or to android market to download it)
  • supports USB now and bluetooth in future
  • hardware design toolkit based on Arduino
• android @ home
  • android @ home framework
  • new low cost connectivity protocol for non-wifi devices
  • why not existing??
  • partnering with several industry players
  • LED lights & switches
  • tungsten demo with NFC chips in CDs cannot work unless Google get record label deals…

new devices

• Samsung 10.1 tab: 720p, 565g, 10.1”
  • lighter than iPad 2, at least as powerful…
  • just waiting for better interface

Droidcon London 2010 - Day Two

Whew! When I sat down and looked at the programme for the second day of Droidcon, I wasn’t sure I’d make it until the end — after I’d finished planning out my time, I saw I’d set myself up over 10 sessions I just couldn’t miss.

I’ve now had a day or so to recover so now I’m posting my notes. I hope they’re useful. Please @ me on twitter or leave a comment here if there’s something you’d like to add or correct.

Thanks once again to the organisers of this great event — Dominic Travers, and Kevin McDonagh and Carl-Gustaf Harroch of Novoda. You did a great job and I’m really impressed.

Android: User views

Ilicco Elia @ilicco — Reuters Mobile

Ilicco gave the first keynote speech, with some Reuters promotional video to wake us all up.

• Getting journalists to use mobile as well as working out how consumers use it
• Ilicco spends his time working out how people will use devices before they do
• The App Store was not about the apps…
  • it’s about people
  • it’s about the edge that people believe they will get from the app
• Great apps: * Google Talk — simply excellent * The browser: double clicking a column reflows the text as well as zooming in * Google Music — looking forward to what Google will do here * Vignette Photo app — just so many more styles that the equivalent iPhone apps * New keyboards — swype & swiftkey (great to be able to install device-wide improvements)
• In-app purchasing is seriously lacking
• “I want to buy an iPhone because it’s beautiful”
  • but… “I don’t want a phone that needs a new Mac to activate” !!
  • so… “I just want a normal phone” (HTC Desire)
  • Android can be just a phone — can have multiple personas
  • Great advantage, but also a flaw (not easily pigeonholed)

Growing the value of the application network

Christophe François — Vice President, Mobile Multimedia Products & Services, Orange

Christophe’s follow-up keynote just didn’t have the pizazz of Illico’s — though it was great to see Orange, Vodafone and Sony Ericsson committing so many people and so much time to Android.

• Orange San Francisco Android phone selling for just £99 on PAYG
• Customer survey over 5500 people in four countries (plus 300m customer interactions over 2 years):
  • relevant to their lives
  • security: privacy protected
  • straightforward pricing with no surprises
  • simple services to buidl their own experience
  • their content, when and where they want it
  • easy way to discover and pay for content
• Orange promoting a few focussed apps:
  • Orange TV with premium events
  • Connectivity & customer care
  • News, radio, Orange Map
• Have launched own distribution channel
  • Orange App Store
  • Don’t have to register — purchases go directly on to bill
  • Live in France, UK; out soon in Romania, Poland, Slovakia
• Orange has a strong presence in Africa
  • will be deploying Orange App Store and Widget platform
• Orange Partner Connect
  • register, submit, test & validate apps
  • central app inventory
  • reporting, sales administration & settlement services
  • private beta in Nov 2010 for Android developers

Create Killer Location apps

Alex Housley @ahousley — Rummble

Droidcon London 2010 - Create Killer Location Apps

View more presentations from ahousley.

• Looking about the same as internet was in 1995, but going faster
• 1.97 Billion on internet now (28.8% of world population)
• Location is not mature — it’s still early days but will accelerate quickly
• Location is not a feature: it’s one element of context
• Friend finders have been done to death
• Zinga have good huge business on top of facebook platform
  • similarly, there will be opportunities working with existing big players in location
• If adding game mechanics into a utility app, make sure you reward important behaviour
• “Where there’s a number there’s a game…”
• Android SDK:
  • android.location
  • can get last known location
  • set a LocationListener (can optionally control location providers)
    • can trigger per time or per distance (better for distance)
  • there’s also proximity alerts
• Alternatives:
  • Skyhook — commercial SDK, but open to revenue share
    • quite smart balanced A-GPS
    • have SpotRank
  • W3C Geolocation API via the browser
    • option: enableHighAccuracy: true
• Testing:
  • Create a mock location provider with a text file containing locations
  • Check out /data/app/misc/location/agps/nmea via DDMS-View
• Local data available:
  • crowd source
  • commercial license
  • open source
  • scrape it…
  • APIS
  • Future: open read/write service
• Rummble use:
  • Geonames — to create boundaries for “Local Heroes”
  • GeoPlanet
  • OpenStreetMap
  • Flickr
  • Google Places — some commercial Ts&Cs
  • Facebook Places — not yet a geo API
    • ask for an additional permission
  • SimpleGeo — cloud infrastructure for location data
    • SpotRank — can see heatmaps of crowd density
• Geocoder class in Android has minimal limits
  • need permission.INTERNET
• Use OpenStreetMap for mapping — better than tiles
• Notifications:
  • Android NotificationManager available since Froyo
  • Xtify
    • commercial, 3rd party API
    • have geo-fencing
    • compatible with 1.x upwards
• Measurement
  • assign key local metrics and goals
  • Flurry, Distimo, Google Analytics
• Rummble is based on subjective logic + other trust metrics
• Rummble API
  • people, places, reviews, check-ins
  • tremors — tweets in locations
  • offers
  • available at http://dev.rummble.com

Excellence in the Android User Experience

Roman Nurik @romannurik — Android Developer Advocate, Google

Roman extended his Android Design Tips (slides below) with some additional info on giving great first impressions, and some new prototyping and asset generation tools that have become available.

Android UI Design Tips

View more presentations from Android Developers.

First impressions

• User experience starts somewhere in the Android Market or even in a blog
• UX extends to reviews in market, recommendations, forums, support…
• set great expectations:
  • icon, title (unique & appropriate), description (honest & useful)
  • description:
    • give the user enough info to quickly glance over it
    • don’t need to include all features
    • leave room for screenshots!
• other first impressions:
  • provide screenshots, youtube video
• get the user up and running as soon as possible
  • don’t require a lengthy registration process
  • sync user data and context immediately
• home activity should be simple & straightforward
  • needs user testing on real users
  • highly aesthetic & consistent first impression of brand
• if the UI is more complex, consider a tutorial screen
  • e.g. WinAmp first screen
  • on first open, hide main activity with an overlay PNG
• tutorial screen also useful for introducing widgets & live wallpapers
• if you do have an intro screen, make sure it’s also available in app settings so user can go back

Design tips

• design process:
  • lay out information hierarchy & data model
  • primary UI navigation pattern
  • prototype UI (see later for tools)
• good nav methods:
  • dashboard + action bar
    • big icons + recently updated
  • 3-4 top tabs
  • horizontally sliding top tabs or homogenous content
    • but not for different content — user must know what they’ll get when they swipe
• bad nav:
  • 5+ bottom tabs
  • don’t use options menu for navigation
  • on-screen back button (all Android devices have a hardware back button)
  • tabs that alter the back stack
• prototyping tools
  • paper + pen
  • pencil for firefox — have some Android stencils available
  • adobe fireworks
• design & dev tips
  • full slides available at http://j.mp/androiddesigntips
  • don’t have modal dialogs (make sure back button works)
  • use dp (or sp for fonts), not px for units
  • don’t port UI from other platforms
  • support D-pad & trackball nav (focusable = true)
  • manage activity stack by using right intent flags
  • handle orientation changes
  • localize
• resources can be set up on a resolution specific mechanism
  • drawable, drawable-hdpi, drawable-mdpi, layout-port, layout-land etc
  • Kevin McDonagh had a good session yesterday on creating a great styles file
• make sure your UI elements are selectable
  • use the selector XML properly — with different drawables for different states
  • see frameworks/base projects in Android git for core system assets examples
• icon guidelines: http://j.mp/androidiconguidelines
  • Roman may be working on an update…

Design tools

• New tools: Android UI Utilities
  • http://j.mp/androiduiutils
  • prototyping stencils for Firefox Pencil
  • Android Asset Studio http://j.mp/androidassetstudio
    • generate icons with appropriate effects
  • Photoshop icon templates
• Blog post: http://j.mp/improvingappquality
  • stability, performance, UI

The story of ribot from penny arcade to retail giant

Antony Ribot @ribot

Antony gave a highly personal account of how he and his brother Jerome founded and grew Ribot to the point where they are designing and building Tesco’s grocery apps.

• started Ribot with £10k savings and two people
• now 10 people
• reading: “how to start and run sandwich and coffee shops”

Lessons learned from Future Platforms over the years

Tom Hume @twhume, Future Platforms

• growing pains
  • gone from 2 to 20
  • 5-6 just about at limit of collaboration without confusion
  • 7-8 is inflection point
  • 12-15 can’t organise a night that everyone can make
  • then need someone to run the process for you
  • 50-60 walk into work and see people you don’t recognise..
• you will have problems with staff
  • FP have introduced a much more rigorous hiring process in the last 18 months
  • interviews now includes brainstorming, estimating and other less usual tasks
  • people leave:
    • lack of career path
    • want to do their own things
• service work
  • resented account management, but realise now that you need it
    • need someone to handle the relationship issues between companies
  • your real process is what happens when the shit hits the fan
  • it’s hard to find a service company that doesn’t have aspirations to have its own products
    • it doesn’t work to leave things until the end of the project
    • instead have 1 day in 10 for each person for experiments
    • can get R&D tax credits for this time!
• get a peer group, advisory board (ex-customers)
• use lightweight metrics
  • use half day estimates
  • work out totals at the end of each month and compare to originals
• “Everybody wants to be Steve Jobs: don’t start by being an arrogant prick!”

Android and CouchDB

Aaron Miller, Developer, CouchOne

• started porting CouchDB to Android before he started working at CouchOne
• CouchDB is a non-relational database (NoSQL) that stores JSON documents
• instead of queries, create “views” that allow fast lookup by keys
• all comms over HTTP
• db is highly durable
  • append-only file format
  • survives battery drops out, process killed etc
• can easily transition an app that uses remote CouchDB
• really good at replication: multi-master replication
  • can write to any server
  • can introduce conflicts, but can deal with that
  • really powerful on a phone
    • can sync with a server or with another phone
    • can have multiple DBs on net syncd to a single DB on phone
  • can have continuous integration (via long-running HTTP response)
• on Android, ported the Erlang VM and then CouchDB on top
  • since all Android apps run in separate instances of Dalvik VM, not that expensive to run another VM
  • works with Android service model to shut it down when not needed
• can install from Android Market right now — but still in beta
  • installs quite a few bits on the phone…
  • takes a while to download
• can forward the CouchDB port from the device to your computer and use existing CouchDB tools
  • adb forward tcp:5985 tcp:5985
• can store application and logic in CouchDB as well (just another document!)
  • e.g. WikiLeaks available as a CouchDB instance…
• once you have a CouchDB app on your phone, you can access it even when you’ve got no internet access
• can show new data instantly as it appears
• CouchDB as an Android service — can bind to it from native apps
• DroidCouch is a Java interface library to CouchDB
• in production:
  • there’s not a good way of having your app depending on another app
  • at the moment Android CouchDB is pretty big
  • in the future, will have a slimmer version
• battery life impact of continuous replication?
  • if just watching, then not much (just holding open an empty connection)
  • will stop and only try and look when your device is awake
  • probably want to use scheduled replication in the background, but turn on continuous replication when app in foreground
• if don’t want to replicate whole server-side, can use a filter on replication
• CouchDB has a user model with login
  • validation methods can check user and associated data

Monetize your apps in emerging markets

Chua Zi Yong, SEO MoVend; runs Android developer group in Singapore

• movend provide: * in-app commerce in 10 minutes * DRM services, product management services
• what devices are popular in emerging markets:
  • S40 & S60 (esp in Philippines) — all top 10 devices
  • BlackBerry, though not corporate — students, housewives
    • 1:50 BlackBerry to iPhone in Indonesia
  • lots of MTK Java/Linux devices (mediatek) — knock-off devices
    • generally low-end devices below $150
  • iPhone is not even 1% in China today
• Android has really picked up in Asia Pacific over last 6 months
• Loads of Android devices coming through
  • Open source and free
  • Reduced cost of the handset by having services that charge per month
• support from low cost hardware manufacturers like MTK
  • seeing first ones in Indonesia
• population more familiar with connecting to internet on Nokia phone than on a computer
  • first and only connectivity to internet
  • only gaming console, social network tool, …
• “they probably know your app, but not that they are online”
• payment if different in china, indonesia & thailand:
  • have physical kiosks — people pay in cash for someone to download the app for them
• more than 50% of mobile app payments in Asia Pacific already
• highly fragmented markets & user behaviour
  • sizeable markets: China, Korea, Japan, Indonesia
• highly regulated market place
  • launching an app on China Mobile takes 6-8 months
  • they want to control everything…
  • and money takes similar time to come through…
  • and low cut of profits — at most 50%
• localized payment methods — at least 20 different forms of credit card in China
  • PayPal doesn’t work that well in many of these markets
• but a lot of money available if you get through…
  • hundreds of millions of users
• localize
  • languages…
  • avoid taboo images (e.g. pointed finger is not friendly in Malaysian context)
• casual games don’t work in China
  • more familiar with MMORG with in-game items

Android has a “dude” problem

Belinda Parmar @belindaparmar, Lady Geek TV

• Lady Geek TV: marketing company that helps companies talk to women
• in survey only 5% of women said Android for their next phone, 57% said an iPhone
• BUT… more women than men bought smartphone in the last 6 months
• and more female gamers 25-35 than men
• Forrester did some market segmentation on women gadget owners
  • 37% self sufficient, tech savvy
  • 35% neutral, little engagement, low willingness
  • 28% opportunity
• iPhone is marketed as what it does for your life
• women feel overwhelmed and confused by choice of Android devices
• women are twice as likely to have never downloaded a single app
  • don’t see most of the apps as relevant to their lives
  • want apps to solve a problem, to answer a question
• women are interested in apps but want to be involved on their terms
• Recommendations:
  • solve a problem
  • entertain, don’t educate
  • if you have an app and want it featured in the next season of the App Show, get in touch at http://facebook.com/ladygeektv

Android behind the scenes

Karl-Johan Dahlström and Erik Hellman

The afternoon keynote session: Sony Ericsson was very open about their process in designing, building and improving their Android phones. They’ve got some lovely devices (I bought an X10 Mini Pro for my wife) but are currently behind in Android software — their devices run 1.6 and a 2.1 update has only just been announced. However, they seem really committed to the platform going forwards (and they did make the best of a bad job on J2ME…)

• Sony Ericsson wanting to be the Android benchmark going forward
• have Google base, adding Sony features, then Sony Ericsson design
• have teams going out to cities and imbuing culture to identify macro trends
• current focus: Human Curvature, Precision by Tension
  • fits to your body, but precise interaction
• X10 Mini inspired by the Cloud Gate in Chicago, a “pure object”:

• hardware and software need to harmonise

• Now introducing “(Erik) Hellman, version 2.0 of Hellboy” :-)

• all products using same source code
• contributing all changes back to platform
  • sony ericsson one of the main contributors to Android Open Source project
• decouple apps — signature apps built on top of standard Android Platform
• No point setting up an Android forum solely for Sony Ericsson
  • we should be where you are already, answering questions and providing support
• promotions:
  • have app for the week on local facebook groups
  • Susanna working for marketing in UK — going out to stores, telling people which apps to demo
• launching Ericsson Application Awards 2011
  • theme: connected things
  • prizes: € 60K + phones at various stages of competition
  • two categories: students or companies with less than 100 employees
  • launch —> submissions —> beta-test —> prize ceremony
  • http://ericssonapplicationawards.com

Turn good ideas in to great apps

Reto Meier @retomeier, Developer Advocate for Android, Google

See Beginner’s guide to Android for more details on deadly sins & glorious virtues

Five deadly sins

• sloth
  • responsiveness is crucial
  • avoid modal dialogues & activities
  • always update the user on progress — and hourglass is not enough
  • render the main view and fill in data as it arrives
  • you’re not going to have the user’s full attention
  • as soon as they look away, they’re going to do something else
  • never lock people out of the app while you’re processing
  • have about 5 seconds for UI to respond until the OS pops up a warning
  • that’s a back stop to protect the user from crashing apps
  • UI should respond in 100-200ms
  • use AsyncTask
• gluttony
  • if you’re app uses too many resources, it will get killed when it goes into the background
  • don’t overuse WakeLocks
  • think about how to do battery-draining stuff as infrequently as possible
  • let users manage updates
  • share data
  • use Receivers and Alarms, rather than Services and Threads
  • instead of WakeLocks, add flags to WindowManager (FLAG_KEEP_SCREEN_ON)
• hostility
  • respect user expectations for navigation
  • workflow should be self-explanatory
  • don’t hijack the native experience — other people use the device in different ways than you
  • give users options to do things, especially around network connections & location
• arrogance
  • undocumented APIs may not be supported in future releases
  • save state when app is paused so can restore nicely
  • will put off users if you restrict rotation behaviour
• descrimination
  • APK is designed to work with devices from SE X10 Mini to Galaxy Tab to Google TV
  • include multiple graphic resources
  • ensuring hardware happiness:
    • use uses-features in manifest (and make not required if happy for alternatives)
    • then check for API existence in code to modify UX

Five glorious virtues

• beauty
  • people perceive a beautiful app as more professional
  • e.g. tweetdeck gets installed on devices with a pre-installed twitter app
  • optimize assets
• generosity
  • use intents to start other apps
  • e.g. Where and OpenTable
  • create widgets, live folder, live wallpapers to increase user engagement
• utility
• measure your success
  • user feedback
    • can’t follow instructions
    • can provide unhelpful feedback
    • users LIE :-)
    • instead use analytics (e.g. Google Analytics for Mobile Apps, Flurry)
• be epic
  • change the paradigm

Quick tips

• let the runtime kill your service
  • service should return START_NOT_STICKY
  • if started several times then think about making it sticky
• kill your own service
  • use alarms & intent receivers for periodic tasks
  • use inexact repeating alarms
• cloud to device messaging
  • works best for targeted updates
  • com.android.C2DM package is open source sending mechanism for server

Android beyond the phone; Tablets, eReaders, and more

Al Sutton, Company Director of Funky Android

• Dell Streak uses mDPI resources but has much bigger screen
• android.os.Build gives you device-specific data
• use library projects to keep common core together
• even Facebook and Twitter get it wrong (and have done on Dell Streak)…
• there are Android dual screen displays
  • e-Ink displays behave completely differently
  • to get it to work requires a custom class…
• custom device manufacturers are really keen to have your apps on their devices
  • they’ll expect a 20-50% markdown, but you won’t have to pay app store fees
• ViewSonic ViewPad 7 available in the UK for £399
  • 800x400px display
  • offering £100 rebate for any old laptop or netbook
  • runs Android 2.2 and has Wi-Fi, Bluetooth, GPRS & 3G
  • it sometimes says it’s a phone

Evolving past the fingertip

Nick Hunn, Director, WiFore Consulting (http://www.nickhunn.com)

• Lots of proprietary devices coming out that are essentially connected tablets (or could be)
  • Energy monitors (govt has mandated that we will all get two)
  • Home health hubs
  • Android can do their jobs very easily
  • But it’s much harder on embedded operating systems
• putting sensors into everyday objects
  • blood glucose monitor in toothbrush
  • thermometer in TV remote control
• Bluetooth low energy coming out — new name for WiBree * connectionless, mostly OFF * good at small, discrete data transfers * will appear in phones in middle of 2011 * coin cell will die of old age before bluetooth before it runs out! * costs about $1 in bulk, size of a fridge magnet * chips are 5mm square * not really Bluetooth, just been designed so it’s compatible
• can also do broadcasting
  • can be location — great for indoor location
  • range is about 100m outdoors, 50m indoors (can cut down if you want)
• people keep updating online tracking services (e.g. weight, running, electricity usage) for 4-6 weeks then give up
  • too much hassle
• instead, get devices (scales, pedometers, etc) to update online services via the phone
  • changing the source of ARPU for network operators (lots of small packets of automated data)
  • information is sent automatically, without user touching phone
• often much more compelling ways of showing data than just basic units and graphs
  • some scales are just showing difference since last measurement!
• phones don’t include RFID as it’s not a standard frequency around the world
  • one country’s frequency is actually illegal in other countries
  • NFC is different and starting to appear in phones
  • NFC requires more power than Bluetooth Low Energy
  • starting to see different standards working together instead of fighting

The Snapdragon Mobile Development Platform

Leon Farasati, Senior Product Manager, Qualcomm - leon@qualcomm.com

http://developer.qualcomm.com

• Qualcomm is taking on a new role of being the link in the ecosystem
  • ensuring that there are great apps for the ecosystem
  • have a huge installed base, want to make sure that apps work well
• Snapdragon is a system on a chip
  • ARM-based CPU, GPU, rich multimedia, GPS, 3G, Camera, power management
• offering pre-load opportunities…
• Snapdragon MDP (mobile development platform):
  • commercial-like device (can actually use it in the field)
  • will be available through bsquare
  • full load of sensors including:
    • gyro and compass
    • proximity, temperature, pressure
• early access to next generation hardware

Future of Android Panel

Ewan MacLeod with panel:

• Ben Medlock — TouchType (makers of SwiftKey)
• Volker Hirsch — ScoreLoop
  • mobile social infrastructure tools
  • virtual economy tools, good, gifting, etc
• Aaron Johnson — Principal Product Manager for Games & Applications at Vodafone
• Michael Smith — Head of Platforms for INQ Mobile

• Leon Farasati — Senior Product Manager, Qualcomm

• Anssi Vanjoki likened Android to “peeing in your pants”…

  • MS: Nokia not core of mobile software any more
    • Android opens up ecosystem of competition
    • Allows delivery of applications at web speed
    • Just that speed alone is enough…
  • AJ: there will be a point when people start asking for money for Android (”What!? I’m not sure I understand his fear here. There are licenses in place…”)
  • VH: SMS & Smartphone outside of Western Europe etc, Nokia is still a big player
    • but no longer in mobile developed countries
  • BM: a platform that fosters partnership is better
    • Android has challenges with fragmentation
  • VH: iPhone developers were complaining about 3GS causing fragmentation
    • nothing compared to J2ME: multiply devices by operators by bugs…
  • LF: one challenge for Android is capturing lower end
    • but high end phones will trickle down
  • MS: fragmentation to Android is similar to building your own PC in late 80s
    • i.e. no impact

• We’re still on the dream phase for Android: consumers “only buy one Android device”… Will consumers retreat to “something familiar”?

  • VH: is HTC sense out there because Android needs it or because HTC does?
    • Android lacks the polish at the moment
  • AJ: didn’t like the UI on various Nokias I had, but I kept getting them…
    • there’s whole range of reasons why a customer chooses a device
  • BM: the breadth of Google’s web services provides a very strong disincentive to leave
  • MS: Google is encouraging OEMs & operators to fight amongst themselves to get great user experience
  • LF: want to allow the people who actually know the end users to drive that
    • this is the OEMs & operators (”is it really?”)
    • e.g. Japanese phones are tailored to Japanese market — we would probably hate their UI over here

• If I was your fairy godmother, what would you wish to change in Android?

  • BM: I’d love to see a decent automated testing framework on a range of devices
    • love to see something comprehensive
    • I think this should be Google
  • VH: a working billing infrastructure
  • AJ: developers making sure that their app manifests include clearly defined API access and permissions
    • so can manage lifecycle
  • MS: Google has history of encouraging competition and then eating them
    • would like Google to be a little more open about what they’re aiming at and what they’re not, to provide some reassurance
  • LF: want a better way of getting hardware acceleration support

• What should we know about? What’s really interesting?

  • BM: the tech community
    • there’s a great community of tech blogs ready to test new software
  • VH: entering Google email in device and seeing calendar & picasa just popping up
  • AJ: paid Android submission to developer.vodafone.com
    • everyone can sign up, with charge to bill included
  • MS: cloud to device messaging
    • should destroy BlackBerry BDM very quickly
  • LF: this room right here — Java brings a huge talent pool
    • peer to peer proximity based stuff
    • device contextual awareness: walking, driving, in your purse/pocket

Over The Air Hack - a LEGO robot with an iPhone brain

As promised, here are the details of my Over The Air hack — The Eyes Have It — which won prizes for “Best Hardware Hack” and “Best Use of Other Features” in the overnight hack-a-thon competition. I also took it along to Mobile Monday London’s Demo Night on the following Monday.

The hack was a LEGO Mindstorms robot with an iPhone brain that followed faces in front of it and steered towards them. It performed well on stage, following me as I gestured towards it, just like a small pet.

The hack was composed of two parts: an iPhone app that detected faces in the video feed from the front-facing camera; and the LEGO robot that took instructions from the iPhone and steered accordingly.

Communication

Getting the two parts to communicate was one of the trickiest areas to get right, and caused extra headaches before each presentation. Both LEGO Mindstorms and the iPhone can communicate over Bluetooth, but Apple has restricted Bluetooth communication to companies that will pay the Apple “Made For iPhone” license fee or that use particular hardware. Bluetooth comms is not available through the standard SDK and apparently needs some kind of “secret handshake” to work.

Unfortunately, Mindstorms came out a while before the iPhone and uses a different Bluetooth chip; and LEGO and Apple haven’t managed to do a deal to provide Mindstorms access from the iPhone (perhaps because LEGO has open sourced much of their software?). LEGO has now released an Android app to showcase mobile phone integration, so let's hope Apple can work with them to get some iPhone apps too.

So no Bluetooth, and Apple won’t let you talk over the dock connector either… That left feeding information to the robot through one of its five senses — a touch-sensitive button, an ultrasound distance sensor, a microphone, a light sensor or the motors themselves (which can detect rotation).

A quick search on LEGO Mindstorms iPhone brought up the iPhoneRobot which used the light sensor to pick up different greys on the screen. This was a great start, but it wasn’t quite what I was looking for. For a start, it required the LeJOS firmware on the Mindstorms brick — this is a cut-down Java VM that replaces the built-in LEGO firmware. I’ve left my Mindstorms with the default LEGO firmware as I use it with my 7 year-old son. He’s not quite ready for Java, but can easily understand the Labview-based visual programming that comes with the Mindstorms kit. Secondly, the robot itself wasn’t quite suitable — I wanted a robot that would recognise faces, so I needed the iPhone to be pointing upwards towards people’s heads rather than along the ground.

Motor skills — building the robot

Designing a new robot from scratch takes quite a while and I only had overnight. Unlike normal LEGO with bumps and holes, the Mindstorms kit uses the new-style LEGO Technic, which is mostly holes and connectors. Also, LEGO’s own sample models are pretty complex, as they are built to look like animals or people as well as interact.

Fortunately, I found a great site with designs for quick Mindstorms models that are easier to hack to do what you want. Some of the ideas are amazing — especially a Segway that actually balances! I started with the 3-Motor Chassis and added the distance sensor on the front to prevent crashes. I also added the button sensor on the side to help with starting and stopping the program, as the iPhone was in the way of the program buttons on the Mindstorms block.

Here’s some pictures of the final result, holding the iPhone pointing upwards. I’ll try and generate some build instructions later.

Computer vision — the iPhone app

Humans are exceptionally good at seeing faces. Our brains have been trained from birth to detect and analyse faces very quickly. We can tell which way people are looking from far away and even see faces in random patterns.

Computers have a harder time of it, although recent developments have massively improved what is possible. Companies such as Polar Rose have shown demonstrations of both face detection (finding out where any faces are in an image or video) and facial recognition (matching the detected faces against a database of known images) running in real-time on mobile phones. Unfortunately, their code is not available to an overnight hacker, though they’ve recently been bought by Apple so we may see interesting capabilities in future iPhones.

However, Intel launched an open source project back in 1999 called OpenCV (for Computer Vision) and not only is it still going strong, but the library is easily compatible with the iPhone and has a git repository of a ready-built library. OpenCV is all about providing well-optimised functions for real-time computer vision, so that developers do not have to reinvent the wheel. It includes face detection algorithms and a guy called Roy has posted some examples of how to get face detection working on an iPhone video feed.

Roy’s sample code was written for iOS 3, and iOS 4 provides much easier methods for accessing the video feed from the device. I updated Roy’s code to use the new AVCaptureVideoDataOutput class that provides direct access to uncompressed video frames from any iPhone camera. This bit took a little while longer than it should have done, as the video feed is provided as landscape (you’re recording a video, right?) whereas its preview feed is oriented the same way as the camera. This was not obvious, and made worse by the fact that face detection algorithms do not work when the image is rotated by 90°… There was a point in the early hours of Saturday morning when I thought there would be no face detection at all!

Anyway, following Roy’s recommendations, I scaled down the input image and adjusted the parameters of the OpenCV face detection call. At the moment I transpose the image before sending it to the detection algorithm, but I suspect it would be faster to use a rotated Haar feature set (the bits that the algorithm picks out in each image to match faces). I also didn’t use Roy’s changes to use integer arithmetic rather than floating point — it turns out that the iPhone 4G has enough grunt to cope with the standard OpenCV code.

You can get the iPhone code from my github repository and try it for yourself. Note that it’s hardwired to use the front-facing camera at the moment. If you don’t have an iPhone 4 just change the AVCaptureDevice to point to the ID of the other camera and the rest of the code should still work (though possibly a little more slowly…).

Light and dark — the LEGO program

So now I had a robot base and an iPhone app that could see faces. The next step was to connect the two together using the light sensor.

LEGO provide a drag and drop programming interface for Mindstorms that lets you build up programs using blocks such as “move motor”, “wait for sensor input” and control logic of loops and if/else switches. It’s quite capable and makes simple programs relatively easy, but using variables and arithmetic can be a little cumbersome.

The main issue in getting the robot to drive was calibrating the light sensor, especially when each demo was under different lighting. After a fair amount of tweaking (some just minutes before presenting at Mobile Monday London’s demo night), the best results turned out to be when I crammed a small piece of cardboard into the hinge that held the light sensor onto the iPhone…

You can download the “.rbtx” file from my github repository, but for those who don’t have the LEGO software, the algorithm is essentially:

1. Calibrate the sensor when pointing at the black and white squares to either side of the control square on the iPhone screen
   • The robot prompts for each reading with its display and waits for you to press the button between each sensor reading
   • The program reads the raw values from the sensor and calculates its own scaling values, as the built-in calibration routines turn on the light on the sensor — this works for reading black lines on a white sheet of paper but isn’t so good at reading the backlit screen of the iPhone…
2. Wait for another button press to start the robot moving — so you can step back and make sure your face is in frame
3. Read the raw value of the light sensor, convert it into a value between -90 and +90, and then steer that amount, then repeat
   • The program checks that the light sensor value is within a reasonable range before steering, otherwise the robot tends to go round in tight circles and you have to run round it like a lunatic trying to get your face in the camera frame!
4. When the distance sensor picks up something closer than 6 inches, stop, play a sound and show a beating heart on the display (“I’ve found you!”)
5. Start moving again when the button is pressed

To give you an idea of what this looks like in the Mindstorms NXT software, here’s a picture of the program!

Prizes

Thanks to Monotype for the beautiful poster of Gill Sans Bold Extra Condensed. They were going to give me two, but were happy to swap one for a copy of FontExplorer Pro instead, so I can see my digital fonts presented almost as prettily. Apparently, it’s now available for Windows as well as Mac OS X.