Put a gateway in the model path

When a model call fails in production, the first question is “what did we send, and what came back?” Without a gateway, the answer lives nowhere.

AI Gateway sits between your application and the model providers. It adds logging, analytics, caching, rate limits, retries, and routing without you rewriting every call. You create a named gateway in the dashboard, then point your calls at it.

For Workers AI through the binding, it’s one option on the call:

const result = await env.AI.run('@cf/meta/llama-3.2-3b-instruct', {
  messages: [{ role: 'user', content: prompt }],
}, {
  gateway: { id: 'flaviocopes' },
})

For a third-party provider like OpenAI, you change the base URL your client posts to:

https://api.openai.com/v1
https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai

Same paths, same request bodies, same API key. Only the transport changes. This is why I keep model calls behind a small adapter: production points at the gateway, local experiments go direct, and the calling code never knows the difference.

Lock the gateway down

The gateway URL contains your account ID and gateway name, so it’s guessable. Turn on Authenticated Gateway and send its token in a second header:

headers['Authorization'] = `Bearer ${env.OPENAI_API_KEY}`
headers['cf-aig-authorization'] = `Bearer ${env.CF_AIG_TOKEN}`

Both headers, not one. The provider key authenticates you to the provider. The gateway token authenticates you to the gateway. Send only the provider key and the gateway rejects the request with a 401 before the model ever runs. It’s a confusing failure the first time you hit it, so now you know.

Create separate gateways per environment, so staging noise never lands in production analytics. Keep request IDs and custom metadata free of secrets, because they end up in logs.

The gateway is not an abstraction layer

A gateway does not make every provider look the same. Providers differ in response shapes, error formats, and safety behavior, and the gateway forwards those differences untouched. Validate the model’s response at your application boundary, the same as you would without a gateway.

Route one practice request through a gateway and match its request ID to the gateway log. Open the Logs view, find the request, and read the model, status, latency, and token counts. That trace is the payoff. Your first production incident gets debugged from data instead of guesses.