Chat & streaming

POST https://api.roar-ai.com/v1/chat/completions is drop-in OpenAI-compatible: messages in, completion out. Every model is served one consistent way, fails over automatically if that path is unhealthy, and every request is metered — see how a model is served.

Temperature on newer models

Some newer models run at one fixed temperature and can't be set otherwise. If you send temperature or top_p to one of them, the request is served at the model's own setting rather than refused. The response carries an x-roar-ignored-params header naming what was ignored, and your request log shows it too. Other settings a model can't use, such as top_k or stop, are still refused with a message that names them.

Streaming

Set "stream": true to receive server-sent events. The final data frame before [DONE] carries token usage.

Copystream = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Write a haiku"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

Tool calling

Pass tools (and optionally tool_choice) exactly as you would with OpenAI. Tool calls come back as tool_calls with finish_reason: "tool_calls" — including over streaming — regardless of which backend served the request. Return results as role: "tool" messages.

Copy{
  "model": "claude-sonnet-5-5",
  "messages": [{"role": "user", "content": "Weather in Oslo?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Current weather by city",
      "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    }
  }]
}

Vision

Models that support image input accept OpenAI-style content parts — a data URL or a public image URL:

Copy{"role": "user", "content": [
  {"type": "text", "text": "What's in this picture?"},
  {"type": "image_url", "image_url": {"url": "https://example.com/cat.png"}}
]}

Documents (PDF)

Send a PDF as an OpenAI-style file content part, with the file inline as a base64 data URL. It works with any chat model, including models that cannot read PDFs on their own:

Copy{"role": "user", "content": [
  {"type": "text", "text": "What is the total on this invoice?"},
  {"type": "file", "file": {
    "filename": "invoice.pdf",
    "file_data": "data:application/pdf;base64,JVBERi0xLjcK..."
  }}
]}

For a model that cannot read PDFs, we read the document on our side first, at no charge, and give the model what it can read:

• Pages with text become the text of the page.
• Scanned pages (pictures of paper, with no text in the file) become an image of the page, for models that accept images.
• Blank pages are skipped.

The model you asked for is the only model that reads anything. You pay for what it reads at its normal input rate: the text, plus each page image (typically around 2,000 tokens per page). Models that read documents themselves receive your file as you sent it.

Scanned PDFs need a model that accepts images. If the model you asked for cannot read images, a scanned PDF is refused with a message saying so. Choose a model that supports vision, or send a PDF that contains text.

Only PDFs are read. Send the file inline as file_data: a file_id that refers to an uploaded file is not supported in chat. Password-protected PDFs are refused. Every refusal is a 400 (bad_request) whose param names the content part, such as messages[0].content[1]. A document that takes too long to read is a 422 (document_too_complex): split it, or send fewer pages. If documents are being read for many requests at once, you may get a 503 with retry-after: wait that long and send the request again.