Everything You Measured About Your Cache Is Probably Wrong 🧊

Two things happened close enough together to be worth telling as one story. We enabled an aggressive edge cache interception setting, and the site began returning 500s — not for some routes, for all of them. And separately, while investigating, we found that the command we had been using for months to check whether caching worked could not actually answer the question.

The outage was the expensive one. The measurement error was the one that will happen to you.

Your probe changes the answer

The habit is to check a cache header with a HEAD request, because it is fast and you only want the headers:

curl -I https://example.com/some/path      # HEAD

On some edge networks, a HEAD request is never a cache hit. It is not cached the way a GET is, so the status header comes back as though the resource is uncacheable — every time, for everything, including resources that are in fact being served from cache perfectly. We spent real time "fixing" cache rules that were already working, because our instrument reported failure by construction.

The correct probe discards the body but issues a real request:

# Real GET, headers only, body thrown away.
curl -sS -o /dev/null -D - https://example.com/some/path | grep -i 'cf-cache-status\|age\|cache-control'
 
# Second request tells you more than the first — a MISS then a HIT is the signal.

Generalize past the specific header: verify with the same request shape your users make. A probe that differs from real traffic in method, headers, or protocol is measuring a different system than the one you are asking about.

Caching decisions are made on things you did not intend as inputs

Two more that surprised us:

Rules frequently key on file extension, not on size or content type. We assumed a large binary asset was uncached because of its size; it was uncached because of its suffix. Renaming the same bytes from one extension to another flipped it from never-cached to reliably cached. If you are reasoning about why something is or is not cached, check the actual matching rule rather than the property that seems relevant.

Range requests do not behave like normal ones. Media elements fetch with byte ranges, and a ranged response does not populate the browser cache the way a full response does. So a video can be edge-cached perfectly and still be re-fetched on every playback. Edge caching and browser caching are separate layers with separate rules, and a measurement of one says nothing about the other.

The outage, briefly

The setting we enabled let the edge intercept and serve cached responses earlier in the request lifecycle — a real performance win, and one that assumes a durable place to put things that the framework's caching model expects to exist. Ours was not configured. The result was not degraded caching; it was a hard failure on every request.

Two lessons, neither about caches specifically:

A performance optimization that changes where requests are handled is not a performance change. It is an architectural change with a performance benefit, and it deserves architectural review. We treated a boolean as a tuning knob because it was presented as one.

"Site-wide 500" is what a missing prerequisite looks like. When a single flag takes down every route, the hypothesis is almost never "the flag is buggy" — it is "the flag requires something that is absent." That reframing finds the cause much faster than reading the flag's implementation.

The setting is still off in our configuration, with a comment saying why and what would have to be true to turn it back on. That comment is the actual deliverable from the incident. A flag flipped back to a safe value with no explanation is a trap for whoever next reads the config and thinks it looks conservative.

The habit worth keeping

Before trusting any measurement of infrastructure behavior, ask: does my probe resemble real traffic, and could the act of probing change the result?

For caches the answer is frequently no and yes, in that order. The same question applies to health checks that bypass the load balancer, latency tests from inside the same region, and rate-limit checks with a different API key than production uses. In each, the instrument is subtly not sampling the system you are trying to understand — and an instrument that reports confidently about a different system is worse than no instrument at all.