Streaming

: tokenhub processing

data: {"id":"chatcmpl-8c1f0e2a","object":"chat.completion.chunk","created":1788393600,"model":"deepseek/deepseek-v4-pro","choices":[{"index":0,"delta":{"content":"Hel"}}]}

data: {"id":"chatcmpl-8c1f0e2a","object":"chat.completion.chunk","created":1788393600,"model":"deepseek/deepseek-v4-pro","choices":[{"index":0,"delta":{"content":"lo!"},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-8c1f0e2a","object":"chat.completion.chunk","created":1788393600,"model":"deepseek/deepseek-v4-pro","choices":[],"usage":{"prompt_tokens":412,"completion_tokens":186,"total_tokens":598}}

data: [DONE]

• Usage is on by default: the last chunk before [DONE] carries the usage object with an empty choices array. Send "stream_options": {"include_usage": false} to leave it out.
• A stream is finished only when data: [DONE] arrives. A body that ends without it is truncated, not complete — treat it as an error rather than as a short answer.
• A failure after the headers were sent arrives as one frame whose payload is {"error": …} with no choices, and no terminator follows. The HTTP status is already 200 by then, so inspect each payload for a top-level error before reading it as a chunk.
• Lines beginning with : are keep-alive comments, sent while the provider has not answered yet. Ignore them.
• If your client disconnects mid-stream, AI Hub stops the provider request and logs the request with status 499. Tokens are recorded only if the provider had already reported usage when the stream stopped.

SDK examples

Use your SDK's Chat streaming helper (stream=True in Python or async iterators in JavaScript), and handle both error frames and truncation. Responses and Messages use their own event protocols.

stream = client.chat.completions.create(
    model="deepseek/deepseek-v4-pro",
    messages=[{"role": "user", "content": "Write a haiku about queues"}],
    stream=True,
)
usage = None
for chunk in stream:
    if chunk.usage:
        usage = chunk.usage  # the last chunk: usage and an empty choices list
    if not chunk.choices:
        continue
    print(chunk.choices[0].delta.content or "", end="", flush=True)
print()
print(usage)

const stream = await client.chat.completions.create({
  model: "deepseek/deepseek-v4-pro",
  messages: [{ role: "user", content: "Write a haiku about queues" }],
  stream: true,
});
let usage = null;
for await (const chunk of stream) {
  if (chunk.usage) usage = chunk.usage; // the last chunk: usage and an empty choices array
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}
console.log("
", usage);

Protocol boundaries and recovery

The Chat completion marker does not apply to Responses or Messages. Follow their terminal events and error envelopes in the error reference. A 200 stream can fail after work starts; preserve its request ID and do not automatically repeat generation after a network error.