Cache invalidation
Size a cache for misses as well as hits
Memory is only one part of the budget. Source capacity, refresh concurrency and recovery behaviour determine whether caching actually reduces operating pressure.
In this article
Estimate useful working data
Start with the distinct keys requested during a representative period and the stored size of their values. Include key overhead, metadata and the cache engine's own overhead rather than multiplying payload size alone.
Separate frequently reused entries from one-off requests. A large stream of unique searches can consume memory while producing few useful hits. Caching every result may evict the small set of entries that actually saves work.
Measure the distribution. An average value size can hide a few unusually large reports that dominate memory or network transfer.
Calculate the miss workload
If an illustrative service receives 2,000 reads per second with a 95 percent hit rate, about 100 reads per second still reach the source. At an 80 percent hit rate, that becomes 400. The request volume is unchanged, but source demand has quadrupled.
These simple numbers assume one source read per miss. A cache loader that joins several services can multiply the impact. Measure the real work done by a miss, including retries.
Test a cold start where the hit rate begins near zero. The source must be protected by load limits, staged recovery or another deliberate mechanism. Steady-state savings do not guarantee safe recovery.
Avoid synchronised refresh work
When many popular entries expire together, they can create a sharp load spike. Spreading expiry times can reduce synchronisation. Coalescing concurrent loads for the same key can prevent many callers repeating identical work.
Check the scope of coalescing. A lock inside one application process does not coordinate other instances. A distributed coordination mechanism adds latency and failure modes of its own.
For suitable data, background refresh may preserve responsiveness, but stale values need a bounded policy. A failed refresh should not extend the old entry forever without visibility.
Compare the full operating cost
Include cache service cost, network traffic, engineering maintenance and incident recovery alongside database savings. A cache that saves a cheap lookup but introduces complex invalidation may not be worthwhile.
Choose capacity using measured demand and a defined degradation plan. Record what happens at memory pressure, cache outage and source saturation.
Revisit the decision after traffic patterns change. New filters, tenants or personalisation can increase key variety and reduce reuse. A once-effective cache can become a costly pass-through without any obvious code failure.
Primary sources
Microsoft: cache-aside patternReferences checked 11 September 2026.