AI Gateway

Limits, budgets & errors

Rate limits, spend caps, and the error envelope.

Every error uses the OpenAI-style envelope, so SDK error handling just works:

Error shape

{
  "error": {
    "message": "Rate limit exceeded for key (60 req/min). Retry in 12s.",
    "type": "rate_limit_error",
    "code": "rate_limit_exceeded",
    "param": null
  }
}

What can limit a request

Status	Code	Meaning
`429`	`rate_limit_exceeded`	Requests/min, tokens/min, or concurrency cap hit. Honor the `retry-after` header.
`402`	`budget_exceeded`	A hard spend cap (key, project or account) is used up for the period.
`413`	`request_too_large`	Body over the inference size limit (10MB).
`422`	`guardrail_blocked`	Your account's data guardrails blocked the request (e.g. credit card numbers headed to an external model).
`502`	`upstream_error`	The model could not be reached, including after failover. Safe to retry.

Seeing a limit before you hit it

Every gateway response — streamed included — carries the ceilings we measured for that call, using the same header names the OpenAI SDKs already understand:

Header	Meaning
`x-ratelimit-limit-requests`	Your requests/min ceiling
`x-ratelimit-remaining-requests`	Requests left in the current window
`x-ratelimit-reset-requests`	Time until the allowance is back to full, e.g. `12s`
`x-ratelimit-limit-tokens`	Your tokens/min ceiling, if one is set
`x-ratelimit-remaining-tokens`	Tokens left this minute
`x-ratelimit-reset-tokens`	Time until the token allowance refills

Pace against remaining and you should not see a 429 at all.

A missing header means we are not measuring that limit for you — never that the limit is zero. If no tokens/min ceiling is set on your account, the -tokens headers are absent rather than reported as 0. Treat absence as "no ceiling of this kind", and read the values that are present.

Two things that can refuse a request are deliberately not reported here, because neither has an honest reading in these units:

Concurrency — the cap on requests in flight at once. There is no OpenAI header for it; you will see a 429 with retry-after if you exceed it.
Budgets and credit balance — measured in dollars, on your billing period rather than a per-minute window, and returned as 402, not 429. Mapping a balance onto a token ceiling would be a guess, and you would pace against it. Use GET /client/budget for the real number.

Budgets

Spend caps can be set per key, per project, and account-wide — the tightest one wins. A cap is either blocking (requests get 402 at the limit) or alert-only (email warning, requests keep flowing). Check live budget status with GET /client/budget using a key with usage:read.

A batch settles hours after you submit it, so it is checked against a blocking cap at submission using an estimate of its cost, and that estimate counts against your budget until the job finishes. A batch too large to fit is refused up front rather than discovered halfway through.

Retries

Idempotent handling on your side plus exponential backoff on 429 and 502 is all you need — the gateway has already failed over and retried before a 502 reaches you.