(If the above sounds like gibberish to you, you’re probably not a programmer
and this post won’t be very interesting.)
SpiderOak clients maintain a SSL connection to a Python Twisted Perspective
Broker service to coordinate their actions with the server and with each
To load balance client connections across several Perspective Broker
processes per storage cluster, and route connections from a single public IP to
many storage nodes, we built a proxy server in Erlang. We’ve been running this
in production for several months now.
The design is simple. Erlang/OTP answers the socket, and speaks the
perspective broker protocol just long enough to learn the authentication
credentials the user is attempting to login with. The Erlang server looks up
the user’s assigned storage cluster and node. From there, it simply proxies
the connection (including replaying the authentication sequence) to a Python
Perspective Broker server. After that, it’s a byte-for-byte pass through proxy
The proxy has some added logic to handle connection affinity — multiple
devices for the same SpiderOak user are passed to the same Perspective Broker
This has allowed us to consume fewer public IP addresses (one per proxy
server, instead of one for each storage node) and take advantage of multiple
processors and greater concurrency per storage machine.
Another small benefit is offloading the cost of SSL from the Python
processes. Erlang has it’s own native implementation of SSL (not based on
OpenSSL) which seems to operate with more grace.
This is our first production Erlang/OTP service, and it hasn’t been without
its speed bumps, but these days it’s as stable as any of our other daemons
while handling much greater concurrency and traffic.
Today we’re publishing the code (AGPL3) in case it’s useful to anyone else
(and feedback from the Erlang community is certainly welcome!) It would be
useful to anyone wishing to be able to distribute a Perspective Broker service
across many backend nodes according to user assignment, or perhaps a starting
point for implementing a Perspective Broker server in Erlang. It will likely
require some minor massaging to with your database scheme. Here’s a link to
the tarball: href="https://spideroak.com/dist/spideroak_ssl_proxy.tar.bz2">spideroak_ssl_proxy.tar.bz2