# Chat & streaming

> Chat completions, SSE streaming, tool calling, vision and PDF documents.

`POST https://api.roar-ai.com/v1/chat/completions` is drop-in OpenAI-compatible: messages in, completion out. Every model is served one consistent way, fails over automatically if that path is unhealthy, and every request is metered — see [how a model is served](/docs/models#how-a-model-is-served).

## Temperature on newer models

Some newer models run at one fixed temperature and can't be set otherwise. If you send `temperature` or `top_p` to one of them, the request is served at the model's own setting rather than refused. The response carries an `x-roar-ignored-params` header naming what was ignored, and your request log shows it too. Other settings a model can't use, such as `top_k` or `stop`, are still refused with a message that names them.

## Streaming

Set `"stream": true` to receive server-sent events. The final data frame before `[DONE]` carries token usage.

<CodeCard caption="Python (streaming)">

```python
stream = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Write a haiku"}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")
```

</CodeCard>

## Tool calling

Pass `tools` (and optionally `tool_choice`) exactly as you would with OpenAI. Tool calls come back as `tool_calls` with `finish_reason: "tool_calls"` — including over streaming — regardless of which backend served the request. Return results as `role: "tool"` messages.

<CodeCard caption="Request excerpt">

```json
{
  "model": "claude-sonnet-5-5",
  "messages": [{"role": "user", "content": "Weather in Oslo?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Current weather by city",
      "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"]
      }
    }
  }]
}
```

</CodeCard>

## Vision

Models that support image input accept OpenAI-style content parts — a data URL or a public image URL:

<CodeCard caption="Image content part">

```json
{"role": "user", "content": [
  {"type": "text", "text": "What's in this picture?"},
  {"type": "image_url", "image_url": {"url": "https://example.com/cat.png"}}
]}
```

</CodeCard>

## Documents (PDF)

Send a PDF as an OpenAI-style `file` content part, with the file inline as a base64 data URL. It works with any chat model, including models that cannot read PDFs on their own:

<CodeCard caption="PDF content part">

```json
{"role": "user", "content": [
  {"type": "text", "text": "What is the total on this invoice?"},
  {"type": "file", "file": {
    "filename": "invoice.pdf",
    "file_data": "data:application/pdf;base64,JVBERi0xLjcK..."
  }}
]}
```

</CodeCard>

For a model that cannot read PDFs, we read the document on our side first, at no charge, and give the model what it can read:

- **Pages with text** become the text of the page.
- **Scanned pages** (pictures of paper, with no text in the file) become an image of the page, for models that accept images.
- **Blank pages** are skipped.

The model you asked for is the only model that reads anything. You pay for what it reads at its normal input rate: the text, plus each page image (typically around 2,000 tokens per page). Models that read documents themselves receive your file as you sent it.

**Scanned PDFs need a model that accepts images.** If the model you asked for cannot read images, a scanned PDF is refused with a message saying so. Choose a model that supports vision, or send a PDF that contains text.

| Limit | Per request |
| --- | --- |
| Documents | 5 |
| Size of each document | 8 MB |
| Pages, across all documents | 100 |
| Scanned pages, across all documents | 10 |

Only PDFs are read. Send the file inline as `file_data`: a `file_id` that refers to an uploaded file is not supported in chat. Password-protected PDFs are refused. Every refusal is a `400` (`bad_request`) whose `param` names the content part, such as `messages[0].content[1]`. A document that takes too long to read is a `422` (`document_too_complex`): split it, or send fewer pages. If documents are being read for many requests at once, you may get a `503` with `retry-after`: wait that long and send the request again.

<Note>Legacy `POST /v1/completions` (prompt in, text out) is not served. Send prompt-completion traffic to `POST /v1/chat/completions` with your prompt as a single user message.</Note>