Protect data, cost, and operations

An AI feature spreads data into more places than you think. Take an inventory: what enters prompts, embeddings, vectors, gateway logs, Agent state, D1, and the provider’s systems. Write the list down. One user message can end up in five of those stores from a single request, each with its own retention and access rules.

Then shrink the list. Minimize what you send, separate tenants, redact logs, and set retention. The cheapest data to protect is data you never sent. If the task needs an order summary, send the summary, not the customer record.

Cap cost before the call

Set budgets and rate limits, and enforce them before the model call, not after. This site’s AI endpoint checks a per-visitor counter and a global daily counter in KV, and only then calls the model:

const globalCount = await bumpCounter(env, `global:${today}`)
if (globalCount > 300) {
  return fallbackResponse()
}
const result = await env.AI.run(model, input, { gateway: { id: 'flaviocopes' } })

With the checks in front, the worst-case daily spend is a number I chose, not a number an attacker chose. A cheap model does not protect you from a script sending a million requests. A counter does.

Observe every boundary

Track model usage, gateway requests, Vectorize operations, Agent instances, tool outcomes, latency, and errors, all under one request ID. That one ID, stitched through Worker logs, gateway logs, and tool audit records, is what turns “a user says it charged twice” into a searchable trace. Turn on Workers observability so the logs persist:

{
  "observability": {
    "enabled": true,
    "logs": { "enabled": true }
  }
}

Redaction applies here too. Log the outcome and the request ID, not the prompt body, unless you decided that this store may hold user content.

Keep the off switch

Keep a feature flag or kill switch for generation and for tools, plus a non-AI path for critical work. When the model misbehaves in production, you want one config change that disables generation while the rest of the product keeps working. If a critical flow only works through the agent, an outage at the provider becomes an outage of your business.

Finish by writing the runbook for five scenarios: leaked prompt data, runaway spend, bad model release, poisoned index, and dangerous tool behavior. For each one, how you detect it, the first containment action, and who decides. One page each, written before the incident, because during one nobody writes well.