Queue Example: Absorbing a Burst of Uploads

A queue turns a burst of work into a steady stream that workers can keep up with. This example follows a photo site where 1,000 uploads arrive in one second, and four workers resize them into thumbnails at their own pace.

The setup. Each upload is stored, and a resize message goes on the queue; the upload request returns at once. Four workers each take one message at a time, and a resize takes 250 milliseconds, so each worker finishes four a second and the four together finish 16 a second.

What the queue absorbed. Without it, the upload servers would have had to resize 1,000 images in the same second they arrived, and most uploads would have timed out. With it, uploads finished at once, and the last thumbnail appeared about a minute later.

What the numbers say. The queue's length is the backlog, and backlog divided by the processing rate is the wait: 984 messages at 16 a second is about a minute. That wait is the cost the site accepted. If a minute is too long, eight workers halve it. If uploads kept arriving at more than 16 a second, the queue would grow without end, and adding workers would be the only fix.

How it relates to the concept. The queue held each message until a worker took it, each message went to one worker, and the producer never waited.

So size workers for the average rate and let the queue hold the burst, and watch the queue's length: a backlog that keeps growing means too few workers.