Listened: SE Radio: Joe Armstrong

I listened today to an interview with Joe Armstrong recorded in late 2007. Joe Armstrong created Erlang. This interview took place right as interest in Erlang was rising due to the growth in generally available multicore machines.

In case you’d like to give it a listen as well, this was Software Engineering Radio episode 89.

My takeaways:

• Concurrency and fault tolerance are the heart of Erlang; the two forces together drove it to the actor model. Its functional nature is an accident of its birth as a modified Prolog.
• OTP provides a wealth of tools and ready-to-go patterns for building fault-tolerant systems.
• Erlang happened, not while looking to gin up a new programming language, but while trying to solve a concrete problem with interesting constraints: How can we make it easier to write telephone switching software that can run for ages and never go down?

From memory, here are my more comprehensive (and correspondingly more rambling) notes:

• Reddit gave the impression at the time that Haskell and Erlang were fighting for the future. Eight years later, and the future still isn’t quite here, or Reddit was as poor a predictor of general behavior then as now. ;)

• Erlang arose almost accidentally out of solving problems inherent in plain old telephone service (POTS) switching:

  • You have a ton of different people connected at once, all doing their own thing. Having lightweight processes makes this easy to model.
  • You can’t take the system down. Ever. So you need hot-swapping.
  • You can’t let the system go down. Ever. So you need fault-tolerance.

• Erlang came out of modifying Prolog to have lightweight processes. This was alongside a number of other researchers implementing POTS switching logic in whatever languages they could get their hands on to run on the random VAX UNIX machine they had in the office: Ada, Concurrent Euclid, Smalltalk, ML, C, and Armstrong with Prolog.

• The driving forces behind Erlang were concurrency and fault tolerance. Its actor approach to concurrency was intentional; the functional nature of sequential Erlang, though, is an accident of growing up inside a Prolog (where you can’t unify a term first to one thing and then to something unequal) rather than inside C. Nothing in the actor model would prohibit an actor performing local mutation of bindings.

  • Armstrong points out that assign-once variables are very handy for debugging, though: If you find a bogus value in a variable, then there’s exactly one place in your program where it was bound to that bogus value, vs one of several places where it might have been updated if multiple assignments to the same name were allowed.

• The “let it crash” ethos is a response to specifications’ refusal to spec anything but the happy path. Instead of ad-libbing some rubbish error handling that dwarfs the code you need to write, and likely introduces some errors of its own, you just don’t even bother handling junk input. The worker process dies, its supervisor respawns it, and a report of its abend is written to an error log.

  • You can later review the error log to decide if you want to introduce logic to handle an error case that keeps cropping up, or if it’s so rare that abending and being reborn is still the right approach.

• The ability to actually distribute Erlang across machines came very late. Everyone involved had the sense it would be pretty easy based on how the rest of the system had been implemented, so no-one actually bothered getting around to doing it for a long time.

• Erlang uses its host OS to manage files and sockets and fork over some memory for the Erlang runtime to manage.

  • Erlang’s runtime system is basically an application container or operating system in and of itself.
  • OTP is a framework (like Rails), but rather than being a framework for writing web sites, it’s a framework for writing fault-tolerant systems.
    • Notice how “fault tolerance” keeps coming to the fore!

• Armstrong contrasts shared memory concurrency (a headache) to message passing concurrency (not so bad). He sees Erlang’s greatest selling point as being easy, painless concurrency.

  • Software transactional memory doesn’t enter into this discussion. Was that even a thing in late 2007? Yeah, in Haskell-land, Harris et al. introduced “Composable Memory Transactions” in PPoPP’05, with a couple other papers following in 2006 per the Haskell wiki.)
  • The message passing approach also arises out of the fault tolerance requirements. If machine A crashes, machine B has to have a copy of all needed data to continue, so A needs to send over a copy rather than just a pointer to its own memory.

• Armstrong reckons it unlikely that Akka can get as good of performance as Erlang, as it is unable to alter its abstract machine language to make the core actor model implementation primitives actually performant primitives, vs emulating these using an ill-suited abstract machine:

  • Switch context from one actor to another
  • Send a message
  • Spawn a new process

• Armstrong’s last remarks were on Erlang’s bit-matching DSL. He also asks a good question: Why do regex always work at the byte, rather than bit, level? Why are there no good bit-level regex libraries?

  • I suppose the answer is likely that, unless you are doing systems programming, you don’t much need them, and systems programmers raised on C are unlikely to turn to a regex library to do bit-smashing.