Latency Numbers: Memory, SSD, Network and Distance

Latency numbers are the typical times a computer takes to fetch data from memory, from a solid-state drive, from another machine in the same data centre, and from another continent. Each is hundreds or thousands of times slower than the one before it, and those gaps decide where data should live and how many trips a request can afford.

The source. The figures below come from Colin Scott's 2020 update of a table first popularised by Jeff Dean of Google around 2010. The exact values drift with hardware; the order, and the size of the gaps, does not. A nanosecond (ns) is a billionth of a second, a microsecond (µs) a millionth, and a millisecond (ms) a thousandth.

Reading the scaled column. If reading from memory took one second, fetching a value from another machine in the same building would take over an hour, and a request across an ocean would take more than two weeks. The analogy breaks in one place: a real machine does not sit idle while it waits, because other work overlaps with the wait. The request that waits, though, waits the full time.

What the gaps rule out. A page that makes 20 sequential calls to a database in the same data centre spends about 10 ms on round trips alone, which is fine. The same 20 calls to a database on another continent spend three seconds, which is not. A cache in memory beats a database read by a wide margin because it skips both the disk and the network. A reader far away is best served from a copy of the data near them, because no code change makes light travel faster.

So when a design moves data between memory, disk, the local network and another continent, count the trips: each step down the table costs hundreds of times more than the one above it.