Streaming responses - Yelu AI API

Streaming delivers model output as it is generated, reducing time to first token. Yelu uses Server-Sent Events (SSE) for compatible Chat Completions and Responses requests.

Enable streaming

Set stream to true. For Chat Completions, each data: line contains a chat.completion.chunk. A terminal data: [DONE] marker ends the stream.

curl -N https://api.yelu.ai/v1/chat/completions \
  -H "Authorization: Bearer $YELU_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{"role":"user","content":"Write a four-line poem about reliable APIs."}],
    "stream": true,
    "stream_options": {"include_usage": true}
  }'

const response = await fetch('https://api.yelu.ai/v1/chat/completions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.YELU_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'Write a four-line poem about reliable APIs.' }],
    stream: true,
    stream_options: { include_usage: true },
  }),
});
if (!response.ok || !response.body) throw new Error(await response.text());

const decoder = new TextDecoder();
let buffer = '';
for await (const chunk of response.body) {
  buffer += decoder.decode(chunk, { stream: true });
  const lines = buffer.split('\n');
  buffer = lines.pop() ?? '';
  for (const line of lines) {
    if (!line.startsWith('data: ')) continue;
    const data = line.slice(6).trim();
    if (data === '[DONE]') continue;
    const event = JSON.parse(data);
    process.stdout.write(event.choices?.[0]?.delta?.content ?? '');
  }
}

import json
import os
import requests

with requests.post(
    "https://api.yelu.ai/v1/chat/completions",
    headers={"Authorization": f"Bearer {os.environ['YELU_API_KEY']}"},
    json={
        "model": "gpt-4o-mini",
        "messages": [{"role": "user", "content": "Write a four-line poem about reliable APIs."}],
        "stream": True,
        "stream_options": {"include_usage": True},
    },
    stream=True,
    timeout=(10, 120),
) as response:
    response.raise_for_status()
    for line in response.iter_lines(decode_unicode=True):
        if not line or not line.startswith("data: "):
            continue
        data = line[6:].strip()
        if data == "[DONE]":
            break
        event = json.loads(data)
        print(event.get("choices", [{}])[0].get("delta", {}).get("content", ""), end="", flush=True)

Event shape

data: {"id":"chatcmpl_...","object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant","content":"Clear"},"index":0}]}

data: {"id":"chatcmpl_...","object":"chat.completion.chunk","choices":[{"delta":{"content":" contracts"},"index":0}]}

data: [DONE]

Parser rules

Do not assume network chunks align with SSE lines; buffer until a newline.
Ignore blank lines and non-data: fields you do not use.
Concatenate text deltas in order.
Accumulate tool-call names and arguments across multiple deltas before parsing JSON.
Treat the response as incomplete until the terminal event arrives.
Handle UTF-8 with a streaming decoder so multibyte characters are not split incorrectly.
Cancel the request when the user leaves or the application no longer needs output.

Responses API events

The Responses API emits typed events rather than Chat Completions chunks. Read each event’s type, handle text delta events, and ignore unknown event types for forward compatibility. Store the final response object or usage event if your workflow needs it.

An HTTP 200 only confirms that the stream opened. Errors can still arrive inside the stream or the connection can end before a terminal event. Preserve partial text only if your product explicitly supports partial results.

Production checklist

Set separate connect, idle-read, and total request deadlines.
Disable proxy buffering for SSE routes you control.
Flush downstream data promptly when relaying a stream to a browser.
Bound output with max_completion_tokens or max_output_tokens.
Record time to first byte, time to first token, total duration, finish reason, and completion state.
Retry only if no irreversible downstream action was taken; a restarted stream generates a new response.

​Enable streaming

​Event shape

​Parser rules

​Responses API events

​Production checklist

Enable streaming

Event shape

Parser rules

Responses API events

Production checklist