AI Gateway
Limits, budgets & errors
Rate limits, spend caps, and the error envelope.
Every error uses the OpenAI-style envelope, so SDK error handling just works:
{
"error": {
"message": "Rate limit exceeded for key (60 req/min). Retry in 12s.",
"type": "rate_limit_error",
"code": "rate_limit_exceeded",
"param": null
}
}
What can limit a request
| Status | Code | Meaning |
|---|---|---|
429 | rate_limit_exceeded | Requests/min, tokens/min, or concurrency cap hit. Honor the retry-after header. |
402 | budget_exceeded | A hard spend cap (key, project or account) is used up for the period. |
413 | request_too_large | Body over the inference size limit (10MB). |
422 | guardrail_blocked | Your account's data guardrails blocked the request (e.g. credit card numbers headed to an external model). |
502 | upstream_error | Every candidate backend failed. Safe to retry. |
Budgets
Spend caps can be set per key, per project, and account-wide — the tightest one wins. A cap is either blocking (requests get 402 at the limit) or alert-only (email warning, requests keep flowing). Check live budget status with GET /client/budget using a key with usage:read.
Retries
Idempotent handling on your side plus exponential backoff on 429 and 502 is all you need — failover inside the gateway already retried other backends before a 502 reaches you.