MomoLondon: Cloud Computing & Mobile

A new venue tonight — up on the 9th floor of Google’s Central St Giles office. They even provided good food for the hungry mobile mondayers…

The evening’s focus was on Cloud Computing and Mobile, and the questions seemed to focus mainly on the cloud with not much discussion about how it related to mobile. The chair, Camille, kept the discussion moving along briskly, but we didn’t really get into any depth on any of the subjects. Maybe the audience could smell the food!

The panel

• Chair: Camille Mendler @cmendler
  • Principal Analyst at Informa
• Rob Easton @Cloud_Easton
  • Head of Google Cloud Platform UK & Ireland
  • Eurovision app used App Engine for back-end
• Dr Janko Mrsic-Flogel
  • CEO of Private Planet — personal cloud
  • 7 out of 10 carriers are providing a personal cloud solution
• Caroline Van Den Bergh @GoldenGekko
  • Golden Gekko mobile agency
  • 85% of apps we build are based on Google App Engine
• Jonathan Raper @MadProf
  • CEO at Placr/transportapi & Visiting Professor at City University
  • purchasers of cloud services
  • mostly built on memset

outages & availability

• JR: best service providers are those that communicate well when things go wrong
  • one provider went down and didn’t tell us anything for an hour

looking forward

• RE: google datacentres include a new one in Taiwan at 15 hectares
  • green computing
  • Google can provide very low latency
  • enables reaching out to new business areas such as the film industry
• JR: still talking quite aggregated performance measures
  • can’t ask for X many transactions for Y pounds
  • not that detailed
  • next step may be platform products
• CVDB: mobile enterprise software saving two days a week
  • managing remote teams
• JR: sometimes blame is shifted between telcos & cloud
  • would like to have better tracing
• JMF: Private Planet have a totally green datacentre in Telecity in Holland for T-Mobile

internet of things

• JR: the problem is trading information
  • automatically discovering different feeds & blending them
  • spend 3 weeks integrating code and 3 months arguing over terms & conditions
• CVDB: best successes have been smaller ones
• CM: find my yak in nepal
  • the yak’s don’t talk back — so there’s no privacy issues!
• RE: getting electricity usage feeds every 10 minutes
• CVDB: there has been scaremongering around Google capturing data
• JR: all our data is royalty-free downstream
  • makes it much easier to deal with
• CM: Copenhagen have bike devices to share traffic information
  • provide individual exercise info
  • but also shared with municipality
• JR: Waze has lots of complicated terms & conditions
  • data is available but locked up

the next generation

• CM: has been proved conclusively that 3D gaming improves your cognitive ability
• JR: …minecraft
• JR: it’s still a bit like early 2000 — mobile operator walled gardens
  • want to take down the barriers and bring the data silos together
• CVDB: working with Saga
  • older people have more time, want to delve deeper
  • tablets more popular than mobile phones

data scientists

• CM: shouldn’t we moving towards everyone manipulating their own data
• JR: digital literacy is very shallow, especially in highest decision-making tiers

security issue — data protection

• safe harbour in tatters
• can’t use the cloud at all if we have any secure data
• JMF: Private Planet always has data centre in the specific country
  • but are governments the biggest problem? what about the corporates?
• JR: if you need to do it, in-country encrypted solutions are available
• RE: sure, there are some workloads that Google can’t take on
  • e.g. we won’t let you audit our data centre
  • so can’t work under FCA regulations
  • over time, some of these regulations are going to change
• JR: all have to do as much as we can to help the government make the right decisions
• JR: sometimes the safest place to do something is in the cloud — e.g. running Windows
  • we don’t allow it on any of our machines
  • but it’s great in the cloud!

what if cloud provider company goes bankrupt?

• JR: escrow
• JMF: most agreements done via a channel partner — so that partner will continue the service
  • single party relationship is much more complicated
• RE: cloud business could be bigger than ads for Google
  • SLA penalties are often a percentage of what you pay
  • but if you pay less and less, the SLA means less and less…

is it becoming like supermarket wars?

• RE: Google reduced prices — and if you use an instance more than 25% per month then get a discount
  • terabyte of storage is $26/month
• CM: just like a loyalty card…
  • who will be Lidl and who will be Waitrose?

will we still be waiting in several years?

• JR: overestimate change in the short term and underestimate in the long term — change is stepped
  • sometimes have to wait for the step change in technology or regulation
• CVDB: business are going to grow up quickly
  • first digital generation CEOs will be around in the next few years

third party dependencies

• JR: Apple reviews can be capricious
  • if Apple fall out of love with Dropbox, then that could cause a problem for lots of apps
• JMF: was a huge debate on mobile medical health a while ago
  • then suddenly there was an app that just works
  • and the issues can just disappear
  • gadget-led terms and conditions…?

speed & access?

• RE: check out server density review
  • blog by someone who reviewed lots of providers and switched to Google
• CVDB: Golden Gekko mandate that all backends must be on Google App Engine
  • mainly so that clients can scale

new innovations in infrastructure?

• RE: Google build our own servers
  • apparently they would be the 3rd biggest manufacturers in the world
  • they value new innovation

where is the next growth to make it easier?

• RE: focused on the developer
  • acquired StackDriver to manage cloud capabilities
• JMF: If This Then That
  • making cloud development open to the consumer
  • certain mobile operators are understanding this and joining up devices in the home
• CVDB: finding that parse.com doesn’t suit
  • one size doesn’t quite fit all
  • prefer google’s flexibility

inviting consumers to get more involved

• CVDB: user testing
  • avoiding ivory tower
  • developed O2 Priority Moments
  • built packaged solutions based on it
  • incentivized behaviour also used in enterprise apps
• JR: designed a first location bank
  • your own location storage — that you can choose to share
  • (Ed. see also fireeagle)
  • next generation have a different approach to privacy
• RE: snapchat is a massive google customer
• audience: teenagers using technology completely differently
  • using snapchat to have a whole conversation
• RE: snapchat consumes more IT than any banking industry
  • JDI has 10 petabytes on Google cloud
  • consumers of IT will be well away from the enterprise
  • can have failure at a much cheaper cost — can learn more

what financial services are possible in the cloud?

• RE: restricted by FCA regulations, but other options possible
  • e.g. datasift to search across twitter
  • big challenge to attract IT talent in financial services as there are so many restrictions

skills

• JR: right now it’s hard to hire — ruby on rails developers are hard to find
  • constraint on development right now
• CVDB: couldn’t agree more — mobile skills are in short supply

tools for analytics & loyalty/crm

• looking at how data can be analysed
• wanting to drive revenue per user
• wanting to get the right tools in place to complete the feedback loop
• CVDB: iris solutions company
  • built the backend themselves
  • ?? http://www.iris.co.uk/
• JR: “build it in the cloud, but just don’t tell IT”
• RE: tableau, BIME, etc. working on top of Google Big Query
  • now a lot cheaper

After the panel discussion

As often happens with Mobile Monday, the discussion after the panel was richer than the panel itself. Some discussion points that came up when I was talking to people afterwards included:

• How do you balance keeping data on the device and in the cloud, to deal with intermittent network availability?
• Will the banks get clobbered by more agile personal finance companies that don’t have to worry about so many regulations?
• Why has it taken Google since 2008 to move from basic App Engine web apps to a full cloud platform?
• How will Google make more money from Cloud Computing than from Ads? There’s a big market, but the price is dropping…
• …and on a completely different note, several people were interested in setting up Code Clubs in their local primary schools, but were stretched for time — perhaps they could work together and contribute part-time?

Google I/O 2011

Some brief notes on the Google I/O 2011 keynote this afternoon… It started kind of dull with Android stats that were pretty good but still seemed to be catching up on Apple, then gradually got better and better.

Google really does want Android to be “one OS everywhere” (I guess running “one web” webapps…) and the UI is improving. I look forward to the media content coming across to countries other than the USA, though the lack of any mention of music or movie labels leaves me in some doubt as to when this will happen. The presentations neatly glossed over the time involved in syncing music and movies between devices and the online service — even on a home wired network the data would take a long time to upload. And there was no mention of syncing between devices on the same LAN like Dropbox either.

The big news for me, however, was the accessories and android@home. By making it so cheap to add hardware integration with Android devices, there could be a real explosion of new products and the home automation market could finally break out of eye-wateringly expensive or home hacker territory. I’m really looking forward to more detail on android@home, but there’s nothing released yet…

android updates

• widgets can now be stretched
  • scrollable -> resizeable (just change XML)
• android can now be a USB host
  • keyboards, mice, game controllers
• google tv market open for devs this summer
  • will be based on android 3.1
• icecreamsandwich coming Q4
  • advanced UI
  • APIs to help scale UI
  • action bar able to reconfigure itself for available space

media

• movies and books now available through android market
  • can watch on web or on android device (e.g. android TV…)
  • seems to be US only (like Music)
• google music
  • cache music recently played
  • can “pin” music just like on movies to make available offline
  • downloads in background
  • 20,000 songs can be added to library
  • free in beta…
  • new music app works with android 2.2 and above

hardware

• new devices will receive latest updates for 18 months after first launch
  • (if the hardware allows)
  • guaranteed to come through reasonably quickly
• android open accessory
  • plug in an accessory
  • phone gets taken to app (or to android market to download it)
  • supports USB now and bluetooth in future
  • hardware design toolkit based on Arduino
• android @ home
  • android @ home framework
  • new low cost connectivity protocol for non-wifi devices
  • why not existing??
  • partnering with several industry players
  • LED lights & switches
  • tungsten demo with NFC chips in CDs cannot work unless Google get record label deals…

new devices

• Samsung 10.1 tab: 720p, 565g, 10.1”
  • lighter than iPad 2, at least as powerful…
  • just waiting for better interface

Over The Air 2010

This was my third Over The Air and it more than lived up to the expectations set by the previous two. Billed as “a grass-roots mobile developer event”, Over The Air is an unmissable event on the mobile developer’s calendar — a weekend of fresh presentations from people who are at the cutting edge, mixed with an overnight hack-a-thon competition with prizes awarded in the final presentation.

This year the organisers went for a couple of keynotes (including the major coup of Tim Berners-Lee on the Sunday morning) as well as over 40 seminars on topics ranging from business oriented pitch and product workshops and DIY PR; through introductions to design & prototyping tools such as Flowella and iProcessing; all the way to detailed technical explanations and tutorials such as using Paypal’s mobile SDKs and Bryan Rieger’s excellent presentation on building mobile and desktop compatible web sites (of which more later).

I’ve put my rough notes on the presentations below, but it’s worth checking out the other presentations that were spotlighted on Slideshare and the other blog entries featured in the overtheair twitter stream.

I also entered the hack-a-thon competition and won Best Hardware Hack as well as Best use of “other features”?! for my LEGO Mindstorms robot with an iPhone brain that followed faces. I’ll be posting the code for my hack shortly…

Keynote: the art of emotional design

Aral Balkan — @aral

• when computers made the impossible, possible, features mattered
  • …and it didn’t matter if the interface was hard
• mobile phones are now a commodity, but we’re willing to pay extra for some things
  • “I think Apple is on to something”
• you cannot compete on features any more:
  • when infrastructure is commoditized, the differentiating factor is User Experience
• Le Petit Prince: “perfection is achieved … when there is nothing left to take away”
• you need someone with the authority and responsibility to say “no” to extra features
• “usable” = “edible”
• we want to aim for “magical”
• Aral tries to stay “naive” so that he can understand interfaces from the point of view of an everyday person (not a developer)
• don’t trick the user
• build in “delighters”
  • not required for interaction, but…
• oslo bullet train ticket machine:
  • just a credit card slot
  • exit also has a credit card swipe
  • can get a receipt later online
  • made a difficult decision: knew that fraud rates would go up
  • but worth it for increased usage
• dangers:
  • weakest link is what is not under your control
  • e.g. facebook pulled the plug on Feathers for Facebook (now Feathers Visage) without warning, just as the app was featured on the German and Austrian app store

DIY research workshop – informing the design process

Mark A.M. Kramer — @mamk

• users have different issues to those in the mobile industry
• Mark does “nomadic ethnography” — mainly based on trains :-)
• often finds himself being a participatory observer
• Mobile research methods
  • e.g. woman on circle line instinctively knew when network was available — put the paper down and switched to checking her mail
• iPhone is a good ethnography tool — can take pictures secretly!
• mobile also enables surveys in the field
• culture is changing:
  • mobile is enabling “just in time” — e.g. “just google that”

Mark then showed us various interviews:

• inventor of makerbot and thingiverse:
  • “share or die”
  • integrate game mechanics into whatever you’re doing
• reactable music interface:
  • play along
  • make it fun
  • can really make instruments with accelerometer
• feature phone user from Eastern Europe (no smart phone culture):
  • want something I can read my magazines on (comic books)
  • but want it portable
  • I’ve seen others using iPads etc.
  • social shaping of technology
• 10 year old — what I like about my iPod touch:
  • app developers will become rich if they make something that people like
  • he likes games
  • doesn’t want to pay for things

things to think about:

• eWaste — what about all those unused phones?
• social isolation
• generational differences
• impact on children
• designing for inclusion & accessibility

Orange Mobilise – the mobile volunteering initiative

David Simoes-Brown (Strategy Partner at 100%Open), Jogesh Limbani (Head of Open Innovation at Orange R&D UK)

• Orange using open innovation to encourage ideas to come in to Orange
  • The Wisdom of Crowds, James Surowiecki
  • Cognitive Surplus, Clay Shirky — 100 million hours of adverts watched over one weekend in America, equivalent to the amount of time taken to create wikipedia
• Orange want to help minutes matter
• Tuangou (cloud clout buying) in China
  • gathering people together to negotiate a discount
• Orange want to capture 5 minutes a day from the 3h 45m TV watching for volunteering
• Have previously launched Orange Rock Corps — can earn a ticket to a gig by volunteering
• 1 million people giving 5 minutes = a decade of real time
• http://mobilevolunteering.co.uk:
  • Orange building an app (iPhone to begin with)
  • Gathering ideas
  • Promoting existing apps
• Challenge:
  • can pick an app from suggested ideas
  • contact the author and go
  • OR submit an existing suitable app
• Orange will launch the top three voted apps & actions in October
• Will also launch 7 other apps & ideas as chosen by Orange rather than community
• Voting criteria:
  • You’re on the bus with five minutes before the stop — what can you do?
  • Popular & used often
  • Has a clear social or environmental benefit
  • Practical & attractive to a sponsor charity — needs a backup
  • Looking for a balance of actions:
  • research | lookout | reach out | skill set
  • Also looking for a balance of purposes
• suggest to look at latest & most active rather than top scoring

Rethinking the Mobile Web – a pragmatic look at creating an accessible and inclusive mobile experience

Bryan Rieger @bryanrieger

Bryan gave a fantastic presentation about how to make workable web sites for mobile and desktop use. There’s also a whole load of discussion on slideshare below the presentation about different approaches. The how-to bits of the presentation start from about slide 104, but there’s a load of stats up front to convince you that having a mobile-friendly site is worth it, and that mobile-friendly does not just mean iPhone…

Rethinking the Mobile Web by Yiibu

View more presentations from Bryan Rieger.

• iPhone in US only covers about 6% of the population and only 4% of 5 major EU countries
• Huge impact but not huge penetration (at least of people, but huge percentage of traffic…)
• nice comparison of older (desktop) Macs vs recent mobile phones
• feature phones of today are the smartphones of yesterday
• the old devices don’t suck
• gs.statcounter.com stats from 02/2010
  • lots of Opera Mini in Nigeria
  • lots of BlackBerry in Australia
  • in Feb. the UC proxy browser ate Nokia’s Chinese market share
  • Nokia bought Novarra in March 2010…
• going forward, main browsers in use are WebKit and Opera Mini
  • coming up later are Mozilla Fennec (will be Firefox Mobile), Ovi & UC proxy browsers
• why is it ok to say “this site works best on Safari 4”, when it wasn’t ok to say “this site works best with IE 4…”?
• design “mobile first”
  • @media queries often fail — so taking a desktop site + media queries results in a desktop site on mobile
  • mobile site + @media queries => desktop site
  • not a new idea — just progressive enhancement!
• use screen and handheld types for first stylesheet
• book recommendation: DOM Scripting by @adactio (if jQuery not available…)
• http://tinysrc.net will resize and compress image for you
• infographics may need reinterpretation for smaller screens, rather than just resizing
• a bit of a dream: Bryan would like to extend the <img> tag to have multiple images for different screen sizes
• Q&A: Opera has device stats available from http://www.opera.com/smw/

Viral Survival Kit: Cloud Computing for Highly Scalable Apps

Matt Wood, Amazon Web Services — mawood@amazon.com

A quick overview of Amazon’s cloud services. Useful for working out what you might need. I was kind of hoping for something more technical, but this is good too. Especially useful is the link to the Amazon Simple Calculator, so you can work out how much your requirements might cost.

• Amazon started with two servers (1 db, 1 app server)
• They refactored so that:
  • dev teams could request resources on demand
  • wouldn’t have to worry about scaling & reliability
• AWS is a datacentre abstraction
• can provision a server in about 20-25 seconds (though Windows servers take a little longer due to security certificate isses)
• guardian open platform is delivered using AWS

S3 & CloudFront

• all data is replicated to three different centres
• scaled automatically as demand rises
• can have a reduced redundancy service at lower cost (only 5 9s rather than 11…)
• CloudFront provides CDN

other services

• RDS: relational datastore — MySQL as a service
• EBS: elastic block store
• EMR: elastic map reduce — hadoop on demand
• VPC: virtual private cloud — elastic resources in the cloud appear in your own network through a VPN
• CloudWatch: monitoring, autoscaling
• ELB: elastic load balancer

some detail

• m1.small available for about 8¢/hour
• m1.large has more CPU etc
• t1.micro is even smaller, only 2¢/hour
  • tight RAM constraints (600Mb), but has automatic CPU scaling
• can plan over longer term and get reduced price (0.07¢/hour if you pay $50 upfront)

designing for failure

• can attach an elastic IP address to an instance
• if server fails, can just repoint to another instance in the same datacentre
• also deal with datacentre failures
  • four regions (East Coast, West Coast, Dublin, Singapore)
  • each region has multiple availability regions
• RDS can enable master/slave with an API call

automation

• can pull monitoring data into ganglia
• monitoring can be connected to autoscaling
• can set thresholds for setting up and killing servers
• integrated with availability zones

security

• can have multiple roles with fine grained access
• provide mitigation against DDOS
• default-deny firewall
• see http://aws.amazon.com/security for details
• data stays within regions

Q & A

• Amazon Simple Calculator provides an estimate of costs

What Would Picasso Do — a panel of art and technology

Another brilliant Bryan Rieger production: find six people with interesting ideas and ask them to inspire the audience by talking about what they liked. It worked…

• Bryan Rieger: http://yiibu.com
• Filip Visnjic: http://www.creativeapplications.net/, curator of http://processing.org
• Jason Fields: Head of Multiplatform User Experience at the BBC, also runs http://app.itize.us/
• mills: ustwo
• Matthias Dahlström: lastminute labs
• Tom Hume: Future Platforms

What Would Picasso Do?

View more presentations from Bryan Rieger.

Tom has been thinking about…

• Joseph Beuys: “everyone is an artist”
• Music industry history is minuscule compared to whole history of art
  • we focus on near-sighted things
• Art doesn’t seem to scale
  • engineering efforts can take 100Ks of people
  • what would art equivalents look like
• Constraints feed creativity

mills is enabling creativity…

• granimator: enabling the public to create screen backgrounds
  • invite artists to contribute assets
• making creativity accessible

matthias has been playing with plasticine…

• dali from 1950s: gem-decorated heart with pumping action
• we work on the surface with the gems at the moment
• need to work deeper
• plasticine interfaces — when they’re rough, people want to grab them and play
• Sony has a big blue design book
  • explored wireless headsets in 80s
  • worked in the home, but in shared spaces you would get someone else’s music
  • dismissed at the time, but might now be treated as an interesting feature

Jason has been looking at apps…

• UK is much more advanced than US in area of IPTV & set-top boxes

Filip has been thinking about art…

• linear vs dynamic
• goals vs accidents
• framing / positioning
• interactive vs reactive
• sense of ownership

Q & A

• Are creative apps on iPad/iPhone merely frivolous?
  • e.g. Brushes, Magic Piano
  • TH: 11-point multitouch is not frivolous
  • JF: a device to interact with while watching TV
  • TH: Jeff Minter: apps for Amiga
  • no intermediation — straight onto the screen
• Are there any useful consequences of art-related apps? Can they be applied to productivity-related apps?
  • TH: epic win: a todo list manager that takes game mechanics and gives you experience points for completing todo items…

tweetalondoncab

This wasn’t a presentation on the main schedule, just the cabbie who started @tweetalondoncab talking about where he’s got to and where he wants to go.

• gone from 2 drivers to 120!
• however, can’t cope with more than 3-4 jobs at a time

process

1. customer DMs to @tweetalondoncab
   • may have to come back to them and get further details (inc phone number)
2. the chair account (protected account) advertises the job (removing specifics) with phone number of person in chair
3. drivers get a text alert when tweet sent from chair account
4. driver responds with a phone call and gets job allocated
   • receives phone number and address of pickup
5. chair account sends out that job has been allocated
6. chair account informs customer of their driver’s details
7. driver contacts customer by phone when they’re ready

also have @cabup protected account that provides shared information updates on demand & traffic — added value for drivers who join in

future ideas

• would like to include cab sharing
• still want to advertise to all drivers as it’s their decision to take it
  • first come first served at the moment
  • but would like to add in how long the driver would take to get there
  • especially for higher value jobs (e.g. Victoria -> Gatwick)
• want to attract people who would normally order a private minicab

constraints & ideas

• dealing with 3-4 jobs coming in at once
  • if can see all cabs on map, would be able to tell customers that can’t help within 30 mins
  • driver claiming job is pinch point
  • have a link to a page with how many minutes to be there
  • claim the job automatically
  • could have a form to submit info, so jobs are better structured

Keynote: Sir Tim Berners-Lee

I didn’t get to take many notes during Sir Tim’s keynote as I was busy finishing my hack (of which more in the next post). He was an inspiring speaker in his own way — you could see the passion for his subject shining through.

His main points were about ensuring things are referable, and that the reference can (and should) be done with URLs.

• if an app doesn’t have a URL then it can’t be linked
• it doesn’t get the interestingness — people can’t use it for other things

BarCampLondon7: iPhone Stats

Chris and Simon gave a good description of what it’s like to be popular in the Apple App Store.

• get an angle for the app
  • 0870 talked about in Guardian, twitter, TechCrunch, …
• 0870 download stats available on simon’s website
• marketing from Apple is not the be all and end all
  • and you have no control, so no warning when serious traffic hits…
• top 5/top 10 is the big hit
• when you put out an upgrade, 60-70% of new downloads will hit your app (and your feed)
  • again you have no control of when this happens
  • it’s usually at night (in the UK)
• hosting:
  • simon started on slicehost
  • went down after a couple of hours…
  • moved to rackspace cloud (engineX)
  • their cheapest server — has cost about $10 so far
• Kieran: tune apache so it can handle more connections at once since mobiles will talk for longer
• Ads:
  • good at the beginning but tailed off rapidly
  • Simon looking at premium ad suppliers
  • need 500K impressions/month
  • made $800 this month, but could make more on a premium network
• Q: Is it a point against you if you release your app on Cydia?
  • A: don’t think so, but then haven’t tried
• stats for paid:
  • if you’re lucky you’ll get 1000 a day
  • that will get you into top 25
  • a little more will get you top 10
  • if you’re focussing on a single territory, make it the US!
• iPhone apps are a great marketing tool
  • but probably won’t cover your costs…
  • Nigel saw a company that saw increase on their mobile web site when the iPhone app went live
• subscriptions are rolling:
  • app will warn you that you need to purchase a renewal
• Kieran: in-app purchasing is extremely successful — much more than subscriptions
• Even Nike couldn’t get app fast-tracked….!
• submitting & checking by Apple:
  • even new versions get treated as a new app
  • they only tell you one problem at a time
  • it’s about a 2-minute review — not a QA test
• AppViz — you need it for viewing your stats
  • nikf: there’s some equivalents that run on your iPhone (see markjnet’s review)
• 0.5% of users write a review, and less than that actually write a comment
  • paid apps get more reviews — people have invested
  • people don’t know how to do reviews unless they delete the app
  • appirater is an open source library that will prompt people to put in a review and take you to the right place on the app store
  • definitely put a feedback screen in the app
• putting apps cheaper at the beginning doesn’t seem to make much difference
  • there’s a bit chasm from free to paid, but then once you’ve made it paid there’s much less of a drop-off

Google Developer Day 2008: A Deeper Look at App Engine -- Mano Marks

• goal of best practices: use less quota so pay less money and google have less load :-)
• free preview will always be free:
  • 500Mb storage
  • 2Gb bandwidth
• currently don’t allow you to pay for App Engine, but committed to charging for additional capacity by end of year
  • CPU: 10-12 cents/hr
  • Storage: 15-18 cents/hr
  • etc.
  • if double free preview, can expect about $40 / month
• currently support Python, but others will be coming
  • they know but they’re not telling
• looking to provide large upload/download support, but not sure how yet
  • current limit is 1Mb for file and response size
• currently no SLA
  • working with Cloud Status to monitor — http://www.cloudstatus.com/
• Google don’t put adverts on app engine apps
  • they will make more money on search :-)
  • They also don’t look at your data

storing data

• keys are limited to 500 bytes
• can’t change the ID or key_name
• transactional read & write with get() and put()

counters

• Bigtable doesn’t know size of tables — that would be O(N)
• Model.count() is a big transaction
• could create an entity that maintains the count
  • frequent updates can cause high contention
  • fundamental limitation of distributed systems
• instead, create sharded counters
  • randomly break counters into shards using a counter config to hold references to all shards
  • when want to count, ask counter config to add them all together
  • use get_or_insert() to fetch or create atomically

memcache

• when you add things to memcache, you define the staleness that you’re happy with
• use memcache to reduce storage and processing requirements

Q & A

• can use urlfetch to request data from your own servers

  • all app engine requests must complete in 10secs
  • and urlfetch must complete faster (4secs?)
  • you get an HTTP error that you can handle if the request times out

• Is there a profiling tool for app engine or Bigtable?

  • not aware of anything
  • difficult to see return on implementing memcache

• What logging is there?

  • there is a log; it logs requests and you can write to it

• SLA…?

  • the quota will be calculated using a moving average, not a total for the month
  • however, the aim is that if you get slashdotted you’ll stay up

• Parallel processes?

  • don’t allow threading
  • don’t allow direct file write access
  • have limited file read access
  • can’t access direct network sockets
  • however, there is a mapreduce implementation for app engine written by a Googler — http://code.google.com/p/httpmr/?

• email restrictions?

  • can send from any developer of app or from a logged in user
  • restricted to sending one per second

• when will Django 1.0 will be included?

  • some people have uploaded django 1.0 themselves
  • django 1.0 includes a C library, so this must be worked around at the moment

• three big languages internally in Google: Python, Java & C++