BarCamp London: Messaging Scales – Matt Biddulph

Asynchronous messaging coming in to web (been around for a long time, esp. in banking & finance)

Concurrent languages: erlang, scala, haskell, e

If everything is done with message passing, there is no need for shared state

messaging systems:

• dopplr uses ActiveMQ (using JSON as message bodies)
• spread is another
• google gears — provides local message bus for thread worker pool

long-running processes:

• e.g. kayak.com travel search, gradually fills in your results

dopplr coincidence calculations:

• local information done immediately
• rest of social network done by passing a message to queue
• workers just collect and get on with it
• if queue is too long can start up some Amazon ECS resources to process the queue for an hour or so

extreme late-binding

• Alan Kay said that he may have overstated the objects in object orientation :-)
• important thing is messages — work can happen anywhere if messages contain context

can have shared state using sequenced messages

• update your own state by sending messages to the bus
• everyone can share the same state
• when go offline can store messages to pass, ready for when reconnect
  • can speculatively update state to present to the user
  • when reconnect can discard speculative state and catch up properly

network quake

• short (200ms) periods of uncertainty, code guesses what will happen in that time before messages come from other players
• guarantee that players will sync about once a second

founder of erlang: “humans are long-running processes, can pass messages, others can ignore”

OTP — built on erlang by Ericsson (who invented the language)

• includes code changes as changes of state
• talks about fault tolerance
• “crash only programming”
  • don’t need shut down — just pull the plug; when start up look for any state to recover
• like XP — a set of practices that work together

Another recommendation: Enterprise Integration Patterns by Gregor Holt

Problems with message passing:

• only do it if the request really is async, otherwise wasting time
• introduces dependencies on messaging system

What’s next? Matt is currently interested in Arduino

• similar to messaging services