Cloudflare AI Gateway: put a proxy in front of your LLM calls

If you call OpenAI from a Cloudflare Worker, at some point you’ll want to see what’s going on. Which prompts ran, which ones failed, how long they took.

You could build logging yourself. Or you can put AI Gateway in the middle and get all of that for free.

I set this up for StackPlan, and in this post I’ll show you how it works. It pairs nicely with the LLM adapter pattern I wrote about — same fetch wrapper, you only change the base URL.

What is AI Gateway?

AI Gateway is a proxy that sits between your app and the model provider.

You create a named gateway in the Cloudflare dashboard (mine is called stackplan). Your requests go to Cloudflare first, and Cloudflare forwards them to OpenAI, Anthropic, Workers AI, or any other supported provider.

The gateway gives you:

• request logs (model, status, latency)
• optional caching of identical requests
• rate limiting
• Authenticated Gateway, so random people can’t spam your gateway URL

Notice that you still pay the model provider. The gateway doesn’t run the model, it just sits in front of it.

Why do we need it?

Without a gateway, every Worker hits api.openai.com directly. That’s fine for a prototype.

But when something breaks in production, you have no trail of what the model returned. With a gateway, you open AI → AI Gateway → your gateway → Logs in the dashboard and you see every call, with status codes and timing.

My advice is to also turn Authenticated Gateway on. The gateway URL contains your account ID and the gateway name, so it’s guessable. With authentication on, Cloudflare rejects any request that doesn’t carry a valid token. It’s one less thing to worry about.

How to create the gateway

In the Cloudflare dashboard:

1. Go to AI → AI Gateway and click Create Gateway
2. Name it something short, like stackplan. The name becomes part of the URL.
3. Turn Authenticated Gateway on
4. Click Create authentication token and save the token — you only see it once

The defaults are fine for everything else. Caching is optional (StackPlan already caches LLM responses in KV, so I left it off).

Point your app at the gateway

Your OpenAI client already posts to {baseUrl}/chat/completions. To go through the gateway, you only change the base URL.

Direct OpenAI:

https://api.openai.com/v1

Through AI Gateway:

https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai

Replace {account_id} with your account ID (run wrangler whoami to get it) and {gateway_id} with the gateway name.

I set this in wrangler.jsonc as a plain var:

{
  "vars": {
    "LLM_PROVIDER": "openai",
    "OPENAI_BASE_URL": "https://gateway.ai.cloudflare.com/v1/{account_id}/stackplan/openai"
  }
}

The OPENAI_API_KEY stays a Worker secret, same as before. The gateway forwards it to OpenAI.

Be careful: you need two headers

This is the part that bit me.

With Authenticated Gateway on, one key is not enough. You need two:

1. Authorization: Bearer <OPENAI_API_KEY> — this authenticates you with OpenAI
2. cf-aig-authorization: Bearer <CF_AIG_TOKEN> — this authenticates you with the gateway

If you only send the OpenAI key, the gateway rejects the request with a 401 (AiGatewayError, code 2009) and the model never runs. I lost some time on this one.

Store the gateway token as a Worker secret:

npx wrangler secret put CF_AIG_TOKEN

Then add the header in your fetch call:

const headers = {
  Authorization: `Bearer ${apiKey}`,
  'Content-Type': 'application/json',
}

if (gatewayAuthToken) {
  headers['cf-aig-authorization'] = `Bearer ${gatewayAuthToken}`
}

const res = await fetch(`${baseUrl}/chat/completions`, {
  method: 'POST',
  headers,
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: prompt }],
  }),
})

I wire gatewayAuthToken from env.CF_AIG_TOKEN, and I leave it unset when calling OpenAI directly in local experiments. The header is only added when the token exists, so the same code works in both setups.

What stays the same

Everything else. Your prompts, your JSON parsing, your Zod schemas, your rate limits.

Only the transport changes: a different base URL, plus one extra header.

This is why I like keeping LLM calls behind a small adapter. My routes never hardcode api.openai.com — they call getAdapter(env) and the environment decides where requests go. Production points at the gateway. Local dev points somewhere else. The code doesn’t change.

When to skip it

For one-off scripts and local experiments, the gateway is overhead you don’t need. Call the provider directly.

For a production Worker that runs AI features for real users, my advice is to put the gateway in front from day one. It costs you one URL change and one secret, and the first time a model call fails in production you’ll be glad the logs are there.