Autoscaling: Capacity That Follows Traffic

Autoscaling changes the number of servers in a tier automatically, by a rule you write, so capacity follows traffic through the day. It adds servers when a measure of load climbs, removes them when it falls, and works only for a tier that already scales out: stateless copies behind a load balancer.

The rule. A common rule is target tracking: keep the average processor use across the tier near a target, such as 50%. If use climbs above it, the autoscaler adds servers; if it falls well below, it removes them. A minimum and a maximum count bound it, so it never drops to zero or grows without limit.

The delay. Between the climb and the new capacity, three things take time. The metric is gathered over a period, often a minute or more. A new server has to start and pass its health checks. And the autoscaler waits for a warm-up before counting the new server's numbers: on AWS EC2 Auto Scaling, that warm-up defaults to 300 seconds when it is not set.

What it buys. You pay for seven servers only during the evening peak, and two at night, instead of seven all day. A failed server is replaced by the same mechanism that adds new ones.

What it needs. Stateless servers, a load balancer that registers new copies, and a rule tuned so the tier grows before it is overloaded. A target of 50% leaves half of each server free to absorb the minutes before new capacity arrives.

So use autoscaling to follow traffic that rises and falls over hours, and remember that every change it makes arrives minutes after the load that caused it.