Our web site cloudno.de runs as a Node.js app among all the other apps hosted on Cloudnode. Our top goal is to provide excellent performance and maximum uptime for all apps including our own.
When a Node.js application is running at Cloudnode it is started as a dedicated server process using its own port. A reverse proxy routes requests targeted at the app to its corresponding port. Thus many apps share a single proxy instance making it a critical component and a single point of failure.
We are using Nodejitsu's node-http-proxy which normally runs stable. But some apps can give it a hard time, because they are experimental or contain resource leaks. When our monitoring detects such a situation, it takes care by restarting the proxy. Everything is fine after that, but the restart causes a short downtime for all apps which are served by that proxy. Until now...
High Availability Proxy To The Rescue
Since 33 days we are running two proxies behind a balancer, which directs traffic through the other proxy during restarts.
The admin panel shows the following statistics for the last 33 days:
- proxy01 has been restarted 5 times with a combined downtime of 09:48 minutes
- proxy02 has been restarted 2 times with a combined downtime of 03:53 minutes
The overall downtime is zero over the period of 33 days 22 hours.
This combination works fantastic, supports websockets, keeps away bad traffic and scales to as many proxies and servers as are needed to handle Cloudnode's future traffic.