Caching: Keeping Frequent Reads Close

A cache is a fast store, usually held in memory, that keeps copies of values the system reads often. A read that finds its value there skips the slower database entirely. The price is freshness: a copy can fall behind the original, and every cache has to decide how far behind it may be.

An analogy, and where it breaks. A cache works like a desk beside the library stacks: the books you use most stay on the desk, and you walk to the shelves only for the rest. The analogy breaks on one point. A library book does not change while it sits on your desk, but the original of a cached value can change in the database while the copy stays as it was.

Hits, misses and the hit rate. A read the cache can answer is a hit; one it cannot is a miss, and goes to the database. The hit rate, the share of reads that hit, is what a cache is judged by. A cache helps most when a small set of values is read far more often than it changes, such as a product page, a user's profile or a site's settings.

Where it sits. Most web systems run the cache as a separate server that every application server shares, such as Redis or Memcached, so a value cached by one server is a hit for all of them. A cache can also live inside each application process, which is faster still but holds a separate copy per server.

When it is not the answer. A value read once, or one that changes on almost every read, gains nothing: the hit rate stays near zero and the cache adds a round trip to every request. Nor does a cache fix a slow write, since writes still go to the database.

So add a cache where the same values are read far more often than they change, and decide from the start how stale each copy may be.