How to stream LLM responses with server-sent events

An LLM can take several seconds to finish an answer.

If you wait for the complete result, the interface looks stuck. If you stream the response, the user starts reading while the model keeps generating.

The model does not finish sooner. Streaming improves perceived latency by showing partial output as it arrives.

We will build the complete path:

browser -> your server -> model provider
browser <- SSE stream  <- SSE stream

Your API key stays on the server. The browser receives only the events it needs.

If SSE itself is new to you, read Server-Sent Events first. The free HTTP course covers response headers, bodies, caching, and connection behavior.

What is being streamed

Models generate text in tokens, but an API chunk is not guaranteed to contain exactly one token.

One event can contain part of a word, one word, or several tokens. Network chunks are different again. One reader.read() can return half an SSE event or five events together.

Keep these layers separate:

model tokens != provider events != network chunks

Your UI should append text deltas in order. It should not assume any useful boundary inside a delta.

The SSE wire format

An SSE response uses the text/event-stream content type.

Each event is made of fields and ends with a blank line:

event: delta
data: {"text":"Hello"}

event: delta
data: {"text":" there"}

event: done
data: {}

The event field names the event. The data field carries its payload.

A complete SSE parser also understands comments, id, retry, multiple data lines, and both LF and CRLF line endings. Our LLM client only needs named events and JSON data, but it still has to buffer incomplete network chunks.

Keep the provider key on the server

Never call a paid model API directly from browser code with your secret key.

The browser can send the prompt to your server:

await fetch('/api/chat', {
  method: 'POST',
  headers: {
    'content-type': 'application/json'
  },
  body: JSON.stringify({
    prompt: 'Explain DNS in plain English'
  })
})

Your server validates the input, adds the credential, and opens the upstream stream.

Proxy the provider stream

Here is a Web Request and Response handler using OpenAI’s Responses API. The same pattern works in server runtimes that support fetch and ReadableStream.

export async function POST(request) {
  const body = await request.json().catch(() => null)
  const prompt = body?.prompt

  if (
    typeof prompt !== 'string' ||
    prompt.trim().length === 0 ||
    prompt.length > 4000
  ) {
    return Response.json(
      { error: 'Prompt must contain 1 to 4000 characters' },
      { status: 400 }
    )
  }

  const upstream = await fetch('https://api.openai.com/v1/responses', {
    method: 'POST',
    headers: {
      authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
      'content-type': 'application/json'
    },
    body: JSON.stringify({
      model: 'gpt-5.4',
      input: prompt,
      stream: true
    }),
    signal: request.signal
  })

  if (!upstream.ok || !upstream.body) {
    const detail = await upstream.text()

    console.error('Model request failed', {
      status: upstream.status,
      detail
    })

    return Response.json(
      { error: 'The model request failed' },
      { status: 502 }
    )
  }

  return new Response(upstream.body, {
    headers: {
      'content-type': 'text/event-stream; charset=utf-8',
      'cache-control': 'no-cache, no-transform',
      'x-content-type-options': 'nosniff'
    }
  })
}

The server does not collect the complete answer. It forwards the upstream ReadableStream as the response body.

request.signal lets the upstream request observe cancellation when the browser disconnects, if the server runtime propagates it. Test this in the platform you deploy to.

The no-transform cache directive asks intermediaries not to rewrite the response. Compression and proxy buffering can otherwise delay small chunks.

Notice that the detailed provider error goes to server logs. The browser receives a generic message. Provider responses can contain request details you do not want to expose.

Read a POST stream in the browser

The browser EventSource API is built around GET requests. Chat normally uses POST because the prompt belongs in the request body.

Use fetch() and read the response stream:

<form id="chat-form">
  <label for="prompt">Message</label>
  <textarea id="prompt" required></textarea>
  <button>Send</button>
</form>

<button id="stop" type="button" disabled>Stop</button>
<pre id="output"></pre>

Create the request when the form is submitted:

const form = document.querySelector('#chat-form')
const prompt = document.querySelector('#prompt')
const output = document.querySelector('#output')
const stop = document.querySelector('#stop')
let activeRequest

form.addEventListener('submit', async event => {
  event.preventDefault()

  activeRequest?.abort()
  activeRequest = new AbortController()
  output.textContent = ''
  stop.disabled = false

  try {
    await streamAnswer(prompt.value, output, activeRequest.signal)
  } catch (error) {
    if (error.name !== 'AbortError') {
      output.textContent += '\nThe answer could not be completed.'
      console.error(error)
    }
  } finally {
    stop.disabled = true
    activeRequest = undefined
  }
})

stop.addEventListener('click', () => {
  activeRequest?.abort()
})

Now implement streamAnswer():

async function streamAnswer(prompt, output, signal) {
  const response = await fetch('/api/chat', {
    method: 'POST',
    headers: {
      'content-type': 'application/json'
    },
    body: JSON.stringify({ prompt }),
    signal
  })

  if (!response.ok || !response.body) {
    throw new Error(`Chat returned HTTP ${response.status}`)
  }

  const reader = response.body.getReader()
  const decoder = new TextDecoder()
  let buffer = ''
  let pendingCarriageReturn = false

  while (true) {
    const { value, done } = await reader.read()
    let text = decoder.decode(value, { stream: !done })

    if (pendingCarriageReturn) {
      text = `\r${text}`
      pendingCarriageReturn = false
    }

    if (!done && text.endsWith('\r')) {
      text = text.slice(0, -1)
      pendingCarriageReturn = true
    }

    buffer += text.replace(/\r\n/g, '\n').replace(/\r/g, '\n')

    if (done && pendingCarriageReturn) {
      buffer += '\n'
      pendingCarriageReturn = false
    }

    const events = buffer.split('\n\n')
    buffer = events.pop() ?? ''

    for (const block of events) {
      const event = parseSseEvent(block)

      if (!event.data) continue

      const data = JSON.parse(event.data)

      if (event.name === 'response.output_text.delta') {
        output.textContent += data.delta
      }

      if (event.name === 'response.failed') {
        throw new Error('The model failed during generation')
      }
    }

    if (done) break
  }
}

The provider sends named events such as response.output_text.delta. We append only the delta field from those events.

Add the small parser:

function parseSseEvent(block) {
  let name = 'message'
  const data = []

  for (const line of block.split('\n')) {
    if (line.startsWith(':')) continue

    const separator = line.indexOf(':')
    const field = separator === -1
      ? line
      : line.slice(0, separator)
    let value = separator === -1
      ? ''
      : line.slice(separator + 1)

    if (value.startsWith(' ')) value = value.slice(1)

    if (field === 'event') name = value
    if (field === 'data') data.push(value)
  }

  return {
    name,
    data: data.join('\n')
  }
}

Multiple data lines belong to the same event, so the parser joins them with newlines.

Use a tested parser in production

The example handles incomplete UTF-8 text, incomplete events, CRLF, LF, a carriage return split across chunks, comments, named events, and multiple data lines.

SSE has more details, including id and retry. Use a tested SSE parser package when you need the complete protocol. Protocol parsing gets less interesting after the first ten edge cases.

The important lesson is not to run this on every raw chunk:

JSON.parse(decoder.decode(value))

A network chunk is not an event. It has no JSON boundary.

Rendering without creating an XSS bug

Use textContent for model text:

output.textContent += data.delta

Do not append model output with innerHTML. Model output is untrusted, even when the model is instructed to return safe HTML.

If your chat supports Markdown, accumulate the text, parse it with a maintained Markdown library, and sanitize the generated HTML before inserting it.

Streaming Markdown also has incomplete states. A code fence or link can begin in one delta and finish later. Rendering the whole accumulated message on each update is safer than parsing each delta independently.

Handle errors before and after streaming starts

Before sending response headers, the server can return a normal HTTP error such as 400, 401, 429, or 502.

After the stream starts, the status is already 200. A later provider failure must travel inside the stream or appear as an unexpected disconnect.

This gives you two error channels:

before first byte -> HTTP status and JSON body
after first byte  -> SSE error event or interrupted stream

Your UI should keep partial text when a stream fails and mark the answer incomplete. Deleting it hides useful context and makes debugging harder.

Cancellation is part of the feature

A Stop button is not just interface polish. It can stop work the user no longer wants to pay for.

The browser aborts its fetch. The server should observe the closed request and cancel the provider request. Some runtimes do this automatically through request.signal; others need explicit stream cleanup.

Test cancellation with a long prompt. Confirm all three layers stop:

1. The browser reader rejects with AbortError.
2. The server request closes.
3. The provider generation is cancelled or disconnected.

If only the browser stops rendering, you may still be paying for generation nobody reads.

Backpressure and slow clients

Streams have backpressure when the consumer reads more slowly than the producer writes.

Forwarding the upstream ReadableStream lets the runtime manage buffering between the two connections. Avoid reading the whole provider response into an array before returning it. That removes the main benefit of streaming and increases memory use.

You still need platform limits. A client that stops reading should not hold a server connection forever. Add an overall deadline and let abandoned requests close.

Production checks

Authentication

If chat is private, authenticate the route before calling the model.

Input limits

Limit request-body size, prompt length, message count, and total conversation size. These are cost and availability controls, not only validation details.

Output limits

Set the provider’s output-token limit. A public endpoint without an output cap can generate expensive answers.

Rate limits

Limit requests per user or API key. IP limits alone can punish shared networks and are easy to rotate around.

Timeouts

Use a deadline for connection setup and another for the complete generation. A stream that emitted one byte should not live forever.

Proxy buffering

Test the deployed path, not only localhost. Reverse proxies and CDNs can buffer small writes. Verify that the first delta reaches a browser promptly in production.

Logging

Log request IDs, duration, provider status, completion state, and usage. Do not log prompts by default if they can contain private data.

When not to stream

Streaming adds protocol parsing, cancellation, partial-state UI, and a second error channel.

Return normal JSON when the output is short or must be validated as one complete object. Classification, moderation, routing, and structured extraction are common examples.

I would stream chat, long explanations, and code generation. I would not stream a boolean, a three-field JSON object, or a background job the user will not watch.

How I would build it

I would first ship the non-streaming request and prove the input, authentication, provider call, and error mapping.

Then I would switch the upstream request to streaming and forward it without changing the rest of the contract. I would add a Stop button before adding Markdown rendering.

Finally, I would test through the production proxy with one slow response, one provider error, and one cancellation.

Streaming is useful when it improves the user’s wait. It is not a goal by itself.

Want me to talk about your product? You can sponsor this site.