Back to roadmap
Module 8 · Caching, Queues, Async WorkDay 07425 min

Hot Keys and Cache Stampedes

When the cache misses, everyone misses at once.

Day 074

Hot Keys and Cache Stampedes

Many clients
client
Cache (miss)
cache
Singleflight
service
Origin
datastore
Signal path
Stampede mitigation flow
Many clients
client
flow
Cache (miss)
cache
Cache (miss)
cache
flow
Singleflight
service
Singleflight
service
flow
Origin
datastore
Memory hook

Hot Keys and Cache Stampedes: when the cache misses, everyone misses at once

Mental model

absorb bursts before they become outages

Design lens

Singleflight introduces brief queueing on the lock.

Recall anchors
SingleflightRandomized TTLSWR

Why it matters

A hot key is a single key with disproportionate traffic. A stampede happens when its cache entry expires and many requests miss simultaneously, hammering the origin.

Deep dive

Mitigations: per-key lock for refill (singleflight); randomized TTL; stale-while-revalidate; pre-warm on deploys.

For unfixable hot keys: replicate the key across N nodes; fan-out reads.

Detect with Redis 'hot keys' tools or sampling.

Demo / scenario

Homepage feed expires every minute, p99 spikes.

  1. Add singleflight: only one filler per missed key.
  2. Use stale-while-revalidate.
  3. Replicate the feed key across 4 cache nodes.
  4. p99 stable through expirations.

Tradeoffs

  • Singleflight introduces brief queueing on the lock.
  • Replication adds maintenance cost.
  • SWR serves slightly stale during refresh.

Diagram

Many clients
Cache (miss)
Singleflight
Origin
Stampede mitigation flow.

Mind map

Check yourself

Loading quiz…

Sources & further reading