Put a gateway in the model path

When a model call fails in production, the first question is “what did we send and what came back?” Without a gateway, the answer lives nowhere.

AI Gateway sits between your application and model providers. It can add logging, analytics, caching, rate limits, retries, and routing without rewriting every application call. You create a named gateway in the dashboard, then point your calls at it.

For Workers AI through the binding, it is one option on the call:

const result = await env.AI.run('@cf/meta/llama-3.2-3b-instruct', {
  messages: [{ role: 'user', content: prompt }],
}, {
  gateway: { id: 'flaviocopes' },
})

For a third-party provider like OpenAI, you change the base URL your client posts to:

https://api.openai.com/v1
https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/openai

Same paths, same request bodies, same API key. Only the transport changes, which is why keeping model calls behind a small adapter pays off: production points at the gateway, local experiments go direct, and the calling code never knows.

Lock the gateway down

The gateway URL contains your account ID and gateway name, so it is guessable. Turn on Authenticated Gateway and send its token in a second header:

headers['Authorization'] = `Bearer ${env.OPENAI_API_KEY}`
headers['cf-aig-authorization'] = `Bearer ${env.CF_AIG_TOKEN}`

Both headers, not one. The provider key authenticates you to the provider, the gateway token authenticates you to the gateway. Send only the provider key and the gateway rejects the request with a 401 before the model ever runs — a confusing failure the first time you hit it.

Create separate gateways by environment and ownership, so staging noise never pollutes production analytics. Keep request IDs and custom metadata free of secrets, because they land in logs.

The gateway is not an abstraction layer

A gateway does not make every provider response identical, so validate the model contract at the application boundary. Providers differ in response shapes, error formats, and safety behavior, and the gateway forwards those differences untouched.

Route one practice request through a gateway and match its application request ID to the gateway log. Open the gateway’s Logs view, find the request, and check model, status, latency, and tokens. That trace is the payoff: the first production incident gets debugged from data, not guesses.