Scalability: Handling More Load by Adding Resources

A system scales when you can handle more load by adding resources, more memory, more processors or more machines, without redesigning it. There are two ways to add them. Scaling up, also called vertical scaling, moves a part to a bigger machine. Scaling out, also called horizontal scaling, runs more machines doing the same job side by side.

What "scales" means precisely. A system scales well when capacity grows roughly in proportion to what you add: twice the servers, about twice the requests a second. It scales badly when adding resources buys little, because something every request shares, often the database, is already the limit. Adding application servers in front of a database at full load adds nothing.

Scaling up. The part keeps running as one machine, now with more processors and memory. No code changes, and nothing new to operate. The limit is the largest machine you can buy, and while that one machine is down, so is the part.

Scaling out. The part runs as several identical copies, and a load balancer spreads requests across them. There is no fixed ceiling: when load grows, add a copy. In exchange the copies must be interchangeable, which means no copy may hold anything the others lack, and there are more machines to watch and update.

Different parts scale differently. Stateless application servers scale out easily, because any copy can answer any request. A database is harder to scale out, because its copies must agree on the data. That is why most systems scale the application tier out early and scale the database up for as long as they can, adding read copies before anything more complex.

Where it is measured. Scalability is checked against the numbers from an estimate: if peak load is 5,200 requests a second and each server handles 2,000, the question is whether a fourth server buys a fourth server's worth of capacity. If the shared database stops it, the system does not scale past that point, whatever the server count.

So for each part of a design, ask how it grows: a bigger machine, more machines, or neither, because something it shares is already the limit.