How Erlang Makes Event-Driven Code Easy

// January 15th, 2010 // Uncategorized

I don’t think it’s a secret that event-driven programming is pretty useful. One of the reasons that I’ve found Erlang so attractive is that it allows you to write code that behaves like evented code, but still feels like imperative code.

To put this in perspective, take a look at Twisted for Python. In order to make a function that waits for something else to occur, you have to split your code into callbacks and then attach them to a deferred object.

from twisted.internet import reactor
from twisted.web.client import getPage
from twisted.python.util import println

def display(page):
  println(page)
  reactor.stop()

getPage("http://www.google.com/").addCallbacks(display)

reactor.run()

In a full-blown Twisted application, you usually are just implementing the callbacks and you pass them to parts of the system that fill in the blanks. It’s quite powerful, but it’s kind of a pain because you end up breaking up your code into these little pieces. The control flow of you program is completely sliced and diced.

Similarly, Event Machine for Ruby has an identical problem.

require 'rubygems'
require 'eventmachine'
require 'pp'

EventMachine::run do
  conn = EM::Protocols::HttpClient2.connect "www.google.com",80
  request = conn.get '/'

  request.callback do |r|
    pp r.content
    EM.stop
  end
end

As you can see, any chain of significant operations ends up being broken into a series of callbacks. While this is quite functional, it rapidly devolves into a mess that makes it very unclear how the code flows. This is probably the number one issue that drove me to Erlang.

So, what does this look like in Erlang?

#!/usr/bin/env escript
main(_) ->
  inets:start(),
  {ok,R} = http:request("http://www.google.com"),
  io:format("~p~n",[R]).

This is interesting to me because it appears to be normal imperative code. Behind the scenes, Erlang sends a message then the “process” sleeps, sends a message to the evented httpc_manager which handles the request, and eventually it receives a response message. In practice, this is very fast and effectively allows you to write code that is shaped normally. However, the chunks of executed code and passed messages exactly map to how a reactor handles deferred function calls.

In C, Java, Python, or Ruby, the main thread would have been blocked. In Erlang, the “process” politely stops and the VM finds another process to run (just like EventMachine or Twisted’s reactors do). Interestingly, Erlang isn’t the only language to get this. You can do something similar in Scala or Go. I find it to be quite powerful and expect to see it a lot more in the future.

Of course, like so many things in Erlang, this is just the beginning. Using the OTP supervision hierarchy makes it easy to tie together systems of this sort of code reliably. It also neatly reduces the load required for safely handling errors, since the supervision tree can neatly replace hundreds of error-handling callbacks (not pictured above). Anyone who has ever had to debug a large event-driven mess will appreciate where I’m coming from. But that’s another post…

2 Responses to “How Erlang Makes Event-Driven Code Easy”

  1. Want to write synchronous code with Twisted? Just decorate all functions that contain deferreds with “@inlineCallbacks”.

    It’s extremely elegant, and the “modern” way most people are writing code with Twisted.

    Here’s some more examples: http://blog.mekk.waw.pl/archives/14-Twisted-inlineCallbacks-and-deferredGenerator.html

    That being said, there are other greater benefits to using Erlang if you are willing to write in a more esoteric / less all-purpose language (IMHO of course).

  2. I pretty much abandoned Twisted for most of my work when twisted.flow was deprecated. It’s good to see that it’s sort of coming back, even if in a more limited fashion.

    This is one of the reasons that I had always wanted yield to be dynamically scoped (not lexically scoped). Since you can’t bury yield in a function (or even hide it with a macro), there’s no way to avoid the nasty idiom of:

    returnvalue = yield somefunction(…)

    If that were the only drawback, I could handle it. Unfortunately, the way this interplays with deferred and the reactor is problematic.

    One of the benefits of Erlang this way is that when you have many levels of calls, the bottom function can schedule itself off (the way that yield is used in generators). While you can emulate the structure of such code in Python, at runtime, it is really just a bunch of deferred-wrapped generators with a chain of callbacks (which roughly correspond to the call stack in Erlang).

    Most tellingly, when a generator throws and exception, your backtrace is remarkably unusable. It’s also very difficult to provide any scheduling functionality or process management on top of these generator chains. It looks the same, but at best it’s a hack that approximates having good support for this in a language runtime. One could credibly argue that Erlang’s solution is a bit overengineered for simple tasks, but the more concurrent and larger the code, the more the payoff.