Chat & streaming
Chat completions, SSE streaming, tool calling and vision.
POST https://api.roar-ai.com/v1/chat/completions is drop-in OpenAI-compatible: messages in, completion out. Every model is served one consistent way, fails over automatically if that path is unhealthy, and every request is metered — see how a model is served.
Streaming
Set "stream": true to receive server-sent events. The final data frame before [DONE] carries token usage.
stream = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user", "content": "Write a haiku"}],
stream=True,
)
for chunk in stream:
print(chunk.choices[0].delta.content or "", end="")
Tool calling
Pass tools (and optionally tool_choice) exactly as you would with OpenAI. Tool calls come back as tool_calls with finish_reason: "tool_calls" — including over streaming — regardless of which backend served the request. Return results as role: "tool" messages.
{
"model": "claude-sonnet-5",
"messages": [{"role": "user", "content": "Weather in Oslo?"}],
"tools": [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Current weather by city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"]
}
}
}]
}
Vision
Models that support image input accept OpenAI-style content parts — a data URL or a public image URL:
{"role": "user", "content": [
{"type": "text", "text": "What's in this picture?"},
{"type": "image_url", "image_url": {"url": "https://example.com/cat.png"}}
]}
POST /v1/completions (prompt in, text out) is not served. Send prompt-completion traffic to POST /v1/chat/completions with your prompt as a single user message.