Streaming

Set stream to true on a chat/completions request to receive tokens as server-sent events (SSE) while they are generated. The gateway forwards the model server's events as they arrive. It adds keepalive comments, adds the cost to the final usage chunk when you ask for usage, and removes that chunk when you do not.

Request

Send stream as the JSON boolean true. The gateway also reads the string "true" and the number 1 as streaming, but SDKs and other OpenAI-style servers expect a boolean. The official OpenAI SDKs set it for you and parse the stream.

bash

curl https://api.bricksum.com/v1/chat/completions \
  -H "Authorization: Bearer sk-bk-..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen3-32b",
    "messages": [{"role": "user", "content": "Hello!"}],
    "stream": true
  }'

python

from openai import OpenAI

client = OpenAI(
    base_url="https://api.bricksum.com/v1",
    api_key="sk-bk-...",
)

stream = client.chat.completions.create(
    model="qwen3-32b",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

javascript

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.bricksum.com/v1",
  apiKey: process.env.KILN_API_KEY,
});

const stream = await client.chat.completions.create({
  model: "qwen3-32b",
  messages: [{ role: "user", content: "Hello!" }],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0].delta.content ?? "");
}

Event format

The response has Content-Type: text/event-stream. Each event is one data: line carrying a chat.completion.chunk whose choices[0].delta holds the new text; the last delta carries finish_reason, and the stream ends with data: [DONE].

text/event-stream

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1710000000,"model":"qwen3-32b","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1710000000,"model":"qwen3-32b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]

Note

Most OpenAI SDKs parse SSE for you when you pass stream=true and yield the delta objects; only raw HTTP clients need to split on blank lines and skip comment lines.

Keepalive comments

Until the model produces its first token, the gateway writes an SSE comment line every 15 seconds so proxies and clients do not time out. This includes the time your request waits in the model server's queue under load. Lines starting with a colon are comments and carry no data; ignore them. A long wait before the first token is normal under load and is not an error.

comment line

: keepalive

Token usage and cost

By default a stream carries no usage. Set stream_options.include_usage to true and the stream ends with one extra chunk whose choices is empty and whose usage object holds the token counts.

request

{
  "model": "qwen3-32b",
  "messages": [{"role": "user", "content": "Hello!"}],
  "stream": true,
  "stream_options": { "include_usage": true }
}

final chunk

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","created":1710000000,"model":"qwen3-32b","choices":[],"usage":{"prompt_tokens":9,"completion_tokens":12,"total_tokens":21,"cost":0.000001155}}

data: [DONE]

The gateway adds usage.cost to that chunk: the amount in USD charged to your credits for this request, computed from the usage counts and the model's per-token prices. Non-streaming responses carry the same usage.cost field. If you do not request usage, the chunk is removed before it reaches you. The token counts and cost in these examples are illustrative figures measured on gpt-oss-120b, not qwen3-32b's.

Every metered response also carries the X-Neocloud-Generation-Id header, the ID of the usage event for this request. Store it with your logs so a charge or an error can be traced to the request.

Disconnects and errors

If your client disconnects mid-stream, generation is not cancelled instantly: the gateway drains the stream to record usage, and the tokens generated are billed.
If the model server fails mid-stream, the error is passed through to you as-is and the request is not billed.
If a stream stops with finish_reason: "length", raise max_tokens, because reasoning tokens count toward it. See API Reference for the known limitations.