Autoscaling Too Slow for a Spike

Autoscaling cannot absorb a spike that arrives faster than a new server can start. Until the new capacity is ready, the servers already running carry all of it, and if they cannot, requests slow and fail. Spare capacity, or scaling ahead of a known spike, covers the gap that autoscaling leaves.

The symptom. A shop announces a sale at noon. At 12:00 traffic triples in under a minute, pages time out for several minutes, and by 12:06 everything is fine, with three times as many servers running.

The trigger. The team relied on autoscaling to handle the sale, with the tier at its usual size at 11:59.

The mechanism. The autoscaler reacts to load it has already measured, and a server takes minutes to start and pass its checks. A rise over an hour gives it time; a rise in seconds does not. For the gap, the four existing servers face three times their load, and overloaded servers do not slow gracefully: queues grow, clients time out and retry, adding still more.

The cost. The sale's first minutes, when the most buyers arrive, are the minutes the site fails.

The remedy. For a spike you know about, scale ahead: schedule the tier to grow at 11:45. For spikes you cannot predict, keep spare capacity, a lower target such as 40%, so the running servers can absorb a sudden rise while new ones start. Making servers start faster narrows the gap but never closes it, and a queue can hold work that does not need an answer at once.

So size the running tier for the fastest spike you must survive, and let autoscaling handle what changes over hours.