Stream answers
Server-sent events for chat completions and agents, with examples.
Streaming sends the answer as it is written, so people see words right away. Both chat completions and agents stream with server-sent events.
Chat completions
Add "stream": true. Each event is a data: line with a chunk in the OpenAI format; the last one is data: [DONE]. Add "stream_options": {"include_usage": true} for a final chunk with tokens and credits.
curl -N https://tengochat.com/api/v1/chat/completions \
-H "Authorization: Bearer $TENGOCHAT_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model": "tengo-1", "stream": true, "messages": [{"role": "user", "content": "Count to five"}]}'data: {"choices":[{"index":0,"delta":{"role":"assistant","content":""}}]}
data: {"choices":[{"index":0,"delta":{"content":"One, two"}}]}
data: {"choices":[{"index":0,"delta":{"content":", three, four, five."}}]}
data: {"choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]Agents
Agents send named events: tool when a tool starts, delta with text, done with the full answer and its thread, and error. See Call your agent from your app for a reader in JavaScript.
Behind a proxy
If your own server passes the stream on, turn off buffering for that route (for nginx, proxy_buffering off;), or the words arrive all at once at the end.