Build persistent streaming chat

A chat UI built on a bare Worker loses everything when the tab closes. The Agents chat layer fixes that by putting the conversation inside a stateful Agent instance. The instance is where the chat lives. The browser is just a window onto it.

The chat agent base class keeps the message history for you. Your job is the model call:

export class ChatAgent extends AIChatAgent {
  async onChatMessage(onFinish) {
    const result = streamText({
      model: this.model,
      messages: buildContext(this.messages),
      onFinish,
    })

    return result.toUIMessageStreamResponse()
  }
}

this.messages is the persisted history, restored from SQLite when the instance wakes. The response streams tokens to the client as they are generated.

The callbacks are not optional

The onFinish handed to onChatMessage is how the completed assistant message gets persisted. Pass it through. Swallow it and the symptom is nasty: the reply streams beautifully, then vanishes on the next page load, because it was never written to storage.

Cancellation matters the same way. A user who stops generation mid-answer should leave behind a consistent history, not a half-written turn that confuses the next model call. Pass the cancellation and finish callbacks the current SDK asks for, so cleanup and persistence both complete.

Bound the context deliberately

Persisted history and model context are two different things. Keep the full transcript in SQLite if the product needs it. But each model call should get a bounded, chosen slice:

function buildContext(messages) {
  return messages.slice(-30)
}

Send everything and long conversations get slower and more expensive every turn, until they exceed the model’s context limit and fail outright. The “chat breaks after 40 messages” bug is almost always this. Thirty recent messages is a fine starting point. Adjust it based on your model’s limit and how long your messages tend to be.

Now test the property that makes this architecture worth it. Disconnect a chat client mid-stream, reconnect it, and check that message state and UI converge without duplicating the assistant turn. Kill the network while a long answer streams, reload, reconnect. The history should show exactly one assistant reply, complete or cleanly truncated. Two copies of the same answer means persistence and streaming are racing each other, and your users on flaky mobile connections will see it daily.