Design for eventual consistency

A KV write can become visible at different times in different locations. The location that performed the write sees it immediately. Everywhere else can keep serving the old cached value for up to 60 seconds.

This surprises people in a second way: even a “missing” result can be cached. If a location recently looked up a key and got nothing, a newly created key may not appear there immediately — the cached miss keeps answering null for a while.

// deploy region:
await env.CONFIG.put('feature-flags', JSON.stringify({ newCheckout: true }))

// another region, moments later:
const flags = await env.CONFIG.get('feature-flags', 'json')
// may still be the OLD flags — or null if a miss was cached

What not to do

Do not retry. Sending more reads does not guarantee that another location’s cache becomes fresh — you get the same cached answer, faster. Retry loops here add latency and cost while changing nothing.

Do not paper over it with sleeps in tests either. A test that passes after sleep(2000) documents a race, not a fix.

Three designs that work

First, make stale acceptable. A feature flag arriving 60 seconds late is invisible to users. Most configuration, preferences, and cached content are in this category once you actually think it through.

Second, version values so staleness is detectable. Include a version or timestamp inside the value, and let the reader decide whether “as of 40 seconds ago” is good enough for this specific action:

await env.CONFIG.put('pricing', JSON.stringify({
  version: 42,
  updatedAt: Date.now(),
  plans: [...],
}))

Third, move the authoritative operation elsewhere. If a decision must see the latest write — a balance check, a uniqueness constraint — perform it in a Durable Object or D1, and optionally publish the result to KV for cheap global reads.

The one rule: the moment of truth must not read KV. Everything around it can.

Now write a test scenario for your own product where one user sees a configuration from 60 seconds ago. Walk through what they experience, and decide whether that is safe or whether that path needs a stronger store.