Stream an LLM Reply to the Browser With Server-Sent Events in Next.js

When a language model answers, waiting for the whole reply feels slow. Streaming it as it is generated makes an app feel immediate. The standard way to do that over plain HTTP is server-sent events (SSE). We use the same idea in an AI tutor for school students, where answers appear word by word. Here is a version you can run in a Next.js app.

What an event stream looks like

An SSE response has the content type text/event-stream and stays open. The server writes messages as text:

event: token
data: {"text":"Hello"}

event: token
data: {"text":" world"}

event: done
data: {}

Each message is a few event: and data: lines followed by a blank line. That is all the protocol is.

Step 1: the Route Handler

Create app/api/chat/route.ts. It returns a ReadableStream as the response body and writes to it as tokens arrive.

// app/api/chat/route.ts
export const runtime = "nodejs";

// Stand-in for your model call: yields a piece of text at a time.
async function* generateTokens(prompt: string, signal: AbortSignal) {
  for (const word of `You asked: ${prompt}`.split(" ")) {
    if (signal.aborted) return;
    await new Promise((resolve) => setTimeout(resolve, 80));
    yield `${word} `;
  }
}

export async function POST(request: Request) {
  const { prompt } = (await request.json()) as { prompt: string };
  const encoder = new TextEncoder();

  const stream = new ReadableStream({
    async start(controller) {
      const send = (event: string, data: unknown) =>
        controller.enqueue(encoder.encode(`event: ${event}\ndata: ${JSON.stringify(data)}\n\n`));
      try {
        for await (const text of generateTokens(prompt, request.signal)) send("token", { text });
        send("done", {});
      } catch {
        send("error", { message: "The reply failed. Try again." });
      } finally {
        controller.close();
      }
    },
  });

  return new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream; charset=utf-8",
      "Cache-Control": "no-cache, no-transform",
    },
  });
}

Three details matter. request.signal aborts when the browser disconnects, so you can stop the model call instead of paying for output nobody will read. Errors are sent as an error event, because the HTTP status has already been sent by then. And no-transform tells proxies not to buffer or compress the stream.

Step 2: read it in the browser

EventSource is the built-in way to read SSE, but it can only make GET requests. A chat prompt belongs in a POST body, so read the stream with fetch:

// lib/stream-chat.ts
export async function streamChat(
  prompt: string,
  onToken: (text: string) => void,
  signal?: AbortSignal,
) {
  const response = await fetch("/api/chat", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ prompt }),
    signal,
  });
  if (!response.ok || !response.body) throw new Error("Request failed");

  const reader = response.body.pipeThrough(new TextDecoderStream()).getReader();
  let buffer = "";

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    buffer += value;

    // Messages end with a blank line; the last piece may be incomplete.
    const messages = buffer.split("\n\n");
    buffer = messages.pop() ?? "";

    for (const message of messages) {
      const event = /^event: (.*)$/m.exec(message)?.[1];
      const data = /^data: (.*)$/m.exec(message)?.[1];
      if (!event || !data) continue;
      if (event === "token") onToken((JSON.parse(data) as { text: string }).text);
      if (event === "error") throw new Error((JSON.parse(data) as { message: string }).message);
    }
  }
}

The buffer is the part people forget. A network chunk can end in the middle of a message, so keep the incomplete tail and add the next chunk to it.

Step 3: use it in a component

"use client";
import { useRef, useState } from "react";
import { streamChat } from "@/lib/stream-chat";

export function Chat() {
  const [reply, setReply] = useState("");
  const controller = useRef<AbortController | null>(null);

  async function ask(prompt: string) {
    controller.current?.abort();
    controller.current = new AbortController();
    setReply("");
    try {
      await streamChat(prompt, (text) => setReply((r) => r + text), controller.current.signal);
    } catch (error) {
      if ((error as Error).name !== "AbortError") setReply("Something went wrong.");
    }
  }

  return (
    <div>
      <button onClick={() => ask("Explain photosynthesis simply")}>Ask</button>
      <button onClick={() => controller.current?.abort()}>Stop</button>
      <p aria-live="polite">{reply}</p>
    </div>
  );
}

Starting a new question aborts the old one, and the Stop button does the same. Aborting on the client closes the connection, which triggers request.signal on the server.

Things to check before you ship

• Limits on your host. Streaming keeps a request open, so look at the maximum duration for functions on your platform.
• Proxies and buffering. Test through your real production setup, not just locally, since some layers buffer responses.
• Accessibility. aria-live="polite" lets screen readers follow the reply without interrupting.
• Cost control. Cancel upstream calls when the user leaves, and cap output length.

Need an AI feature built into your product, with streaming, grounding and cost control? Talk to Paraline Tech.