Streaming Responses
Add stream: true and the gateway relays Server-Sent Events straight from the model as tokens arrive.
Replace YOUR_MODEL_ID with an id returned by GET /v1/models for your key.
curl https://api.exosai.tech/v1/chat/completions \
-H "Authorization: Bearer $EXOSAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "YOUR_MODEL_ID",
"stream": true,
"messages": [{ "role": "user", "content": "Write a haiku about latency." }]
}'The response is Content-Type: text/event-stream. Each event carries a JSON delta in the OpenAI shape, and the stream ends with a sentinel:
data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":"Packets"}}]}
data: {"id":"chatcmpl-…","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" fly"}}]}
data: [DONE]Consuming it in JavaScript
const res = await fetch('https://api.exosai.tech/v1/chat/completions', {
method: 'POST',
headers: {
Authorization: `Bearer ${process.env.EXOSAI_API_KEY}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
model: 'YOUR_MODEL_ID',
stream: true,
messages: [{ role: 'user', content: 'Count to five.' }],
}),
});
if (!res.ok) throw new Error(`HTTP ${res.status}: ${await res.text()}`);
if (!res.body) throw new Error('No response stream');
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = '';
stream: while (true) {
const { value, done } = await reader.read();
if (done) break;
buffer += decoder.decode(value, { stream: true });
const lines = buffer.split('\n');
buffer = lines.pop() ?? '';
for (const line of lines) {
if (!line.startsWith('data: ')) continue;
const payload = line.slice(6).trim();
if (payload === '[DONE]') break stream;
const event = JSON.parse(payload);
if (event.error) throw new Error(event.error.message || 'Upstream stream failed');
process.stdout.write(event.choices?.[0]?.delta?.content ?? '');
}
}