Horizontal Scaling Example: Ten Stateless Application Servers

Scaling out an application tier means running more identical copies of the server and spreading requests across them with a load balancer. This example follows a tier that grows from two copies to ten, and shows what had to be true first and what the growth buys.

The situation. The news site from the estimation topic peaks at about 5,200 requests a second, and one server handles about 2,000. It ran on two servers and failed every evening.

What had to be true first. Every server must be able to answer every request. The team moved sessions from each server's memory into a shared store, and uploads from local disk into object storage. Without that step, adding servers would have logged users out at random.

What it buys. Three servers carry the peak and a fourth covers a failure. On the night of a big story, the team runs ten. Each new copy starts from the same image, registers with the load balancer, passes its health check, and takes traffic; nothing is copied onto it. A failed copy is replaced the same way, and readers never notice.

What it costs. There are now ten machines to deploy to, patch and watch, and a shared session store that every request depends on. The load balancer and the session store each need a spare of their own. And every server still talks to one database: ten servers can send it five times the queries two did.

How it relates to scaling up. The vertical example grew one machine and changed no code. This one changed where state lives, once, and in return can add an eleventh server whenever load calls for it.

So scale stateless servers out: once no server holds anything the others lack, capacity is a count you can raise and lower.