For those who wonder why our hosting service is not yet generally available, we are having scaling issues. The original Nodester ran on one big instance where Node.js servers used a socket/port in a jail and Nodejitsu's node-http-proxy servers where routing the traffic to the servers.
There were a few problems with this approach:
Our capacity was limited to what we could fit on a single machine and the port space was hard limited. Other providers have similar problems, some of them even used public IPs for the servers and ran early into capacity problems. Their way out of this was to keep the servers in an idle state and spin them up when a request hits them.
This was out of question for our service, because our commitment is to provide top performance. So all servers need to be online constantly.
The new infrastructure
The new infrastructure will soon go online and we thought we'd share a bit how it works. The solution is to put the Node.js servers as well as the database servers into docker containers and spin up new containers whenever more capacity is needed. There is no port limit anymore and we can easily scale up by placing containers on new machines. Thanks Docker for the rescue.
Fig 1: New Infrastucture
The routing grid
The original proxies were not flexible enough to route to dynamically changing container addresses. Nginx with a custom Lua router can handle these scenarios and has better performance when serving static content. We came to a similar architecture as Github for their GitHub Pages service using HAProxy and Nginx backed by Redis to store the routing tables. Other than GitHub we chose Redis over MySQL due to its simplicity and easy replication to avoid a single point of failure.
What does change for you?
Docker is 64-bit only and consequently are all Node.js servers now. If you use native modules, make sure to update your app. When you are using a CouchDB or Redis DB on Cloudnode make sure to use the symbolic names of the linked containers from the environment and not IP addresses or even localhost.
Also update your apps to a recent Node.js version as everything below 0.10 will be phased out and cannot be migrated to the new infrastructure.