# An LLM adapter pattern for Cloudflare Workers

> Swap LLM providers with a small adapter pattern. One complete() interface, fetch-based OpenAI calls on Workers, env var switching, graceful degradation.

Author: [Flavio Copes](https://flaviocopes.com/about/) | Published: 2026-07-24 | Topics: [JavaScript](https://flaviocopes.com/tags/js/) | Canonical: https://flaviocopes.com/llm-adapter-pattern/

Your app calls OpenAI today. Tomorrow you want Groq, OpenRouter, or a local bridge in dev. If every route imports a vendor SDK, you're stuck.

[StackPlan](https://stackplan.dev) runs on Cloudflare Workers. Heavy SDKs often **don't run in workerd**. I used a tiny adapter instead.

## One interface

Define what your app actually needs. For StackPlan that's two methods:

```js
/** @typedef {object} LlmAdapter
 * @property {(input: RationaleInput) => Promise<Rationale>} generateRationale
 * @property {(prompt: string) => Promise<string>} completeText
 */
```

Every provider implements the same shape. Routes call `adapter.completeText(prompt)` and parse JSON from the string. They never import `openai` or `@cursor/sdk` directly.

## Fetch-based OpenAI adapter

The OpenAI SDK is fine in Node. On Workers, use `fetch`:

```js
function openAiAdapter({ apiKey, model, baseUrl }) {
  async function completeText(prompt) {
    const res = await fetch(`${baseUrl}/chat/completions`, {
      method: 'POST',
      headers: {
        Authorization: `Bearer ${apiKey}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify({
        model,
        messages: [{ role: 'user', content: prompt }],
        temperature: 0.7,
      }),
    })

    if (!res.ok) throw new Error(`LLM failed (${res.status})`)

    const data = await res.json()
    const text = data.choices?.[0]?.message?.content
    if (!text) throw new Error('Empty LLM response')
    return text
  }

  return {
    async generateRationale(input) {
      const prompt = buildPrompt(input)
      return parseRationaleResponse(await completeText(prompt))
    },
    completeText,
  }
}
```

Any **OpenAI-compatible** endpoint works. Change `baseUrl` to OpenRouter, Groq, or Cloudflare AI Gateway. Same JSON shape.

## Swap providers with an env var

Centralize selection:

```js
function getAdapter(env) {
  if (env.LLM_PROVIDER === 'openai' && env.OPENAI_API_KEY) {
    return openAiAdapter({
      apiKey: env.OPENAI_API_KEY,
      model: env.OPENAI_MODEL || 'gpt-4o-mini',
      baseUrl: env.OPENAI_BASE_URL || 'https://api.openai.com/v1',
    })
  }
  if (env.LLM_PROVIDER === 'cursor' && env.CURSOR_API_KEY) {
    return cursorAdapter(env.CURSOR_API_KEY, env.CURSOR_BRIDGE_URL)
  }
  return null
}
```

Production sets `LLM_PROVIDER=openai`. Local `astro dev` can set `LLM_PROVIDER=cursor` and POST to a Node middleware that runs the Cursor SDK — because that SDK can't execute inside workerd even during dev.

## Degrade gracefully

When `getAdapter()` returns `null`, features should **hide**, not crash.

```js
const adapter = getAdapter(env)
if (!adapter) return null

try {
  return await adapter.generateRationale(input)
} catch (err) {
  console.error('LLM rationale failed:', err)
  return null
}
```

The report page still shows stacks and costs from the rules engine. The prose block simply doesn't render. Prefill from a GitHub URL still works from manifest scanning alone.

Cache successful responses in KV keyed by a hash of the input. Skip the network on repeat views.

The rules engine still picks providers and prices. The LLM only writes prose on top of numbers you already computed. That boundary keeps bad model output from breaking billing logic.

## Parse defensively

Models wrap JSON in markdown fences. Strip fences, find the outermost `{…}`, validate with Zod, retry once on failure.

Keep prompt builders and parsers in the same module as the adapters. The interface stays thin. The messy vendor details stay in one file.

`buildPrompt()` and `buildExtractionPrompt()` live next to `openAiAdapter()`. Routes import `getAdapter` and the parse helpers. Nothing else touches vendor URLs.

## Why not the SDK everywhere

Node apps can use `@anthropic-ai/sdk` or `openai` and call it a day. Workers are different. The runtime has fetch, crypto, and TextEncoder — not the full Node API surface vendor packages expect.

A twenty-line fetch wrapper buys you portability. You test it once. It runs the same in `wrangler dev`, production, and any OpenAI-compatible host.

That's the pattern: one interface, fetch for Workers, env-driven provider swap, null when unconfigured. Your product keeps working when the LLM is off.