Design for eventual consistency

A KV write becomes visible at different times in different locations. The location that performed the write sees it immediately. Everywhere else can keep serving the old cached value for up to 60 seconds.

There’s a second surprise. Even a “missing” result can be cached. If a location recently looked up a key and got nothing, a newly created key may not appear there right away. The cached miss keeps answering null for a while.

// deploy region:
await env.CONFIG.put('feature-flags', JSON.stringify({ newCheckout: true }))

// another region, moments later:
const flags = await env.CONFIG.get('feature-flags', 'json')
// may still be the OLD flags, or null if a miss was cached

What not to do

Don’t retry. Sending more reads does not make another location’s cache fresh. You get the same cached answer, faster. Retry loops here add latency and cost while changing nothing.

Don’t paper over it with sleeps in tests either. A test that passes after sleep(2000) documents a race. It doesn’t fix one.

Three designs that work

First, make stale acceptable. A feature flag arriving 60 seconds late is invisible to users. Most configuration, preferences, and cached content fall in this category once you think it through.

Second, version values so staleness is detectable. Put a version or timestamp inside the value, and let the reader decide whether “as of 40 seconds ago” is good enough for this specific action:

await env.CONFIG.put('pricing', JSON.stringify({
  version: 42,
  updatedAt: Date.now(),
  plans: [...],
}))

Third, move the authoritative operation somewhere else. If a decision must see the latest write, like a balance check or a uniqueness constraint, perform it in a Durable Object or D1. Then, if you want, publish the result to KV for cheap global reads.

The one rule I keep: the moment of truth must not read KV. Everything around it can.

Try this: write a test scenario for your own product where one user sees a configuration from 60 seconds ago. Walk through what they experience, and decide whether that is safe or whether that path needs a stronger store.