Rate limits

Rate limits are enforced per organization. The current defaults are:

Requests: 60 RPM (requests per minute) per organization by default. Exceeding it returns 429.
Tokens: TPM (tokens per minute) is unlimited by default; the operations team can set a per-organization limit. Exceeding it returns 429.
Concurrency: 8 concurrent in-flight requests per organization. While 8 are still running, a further request returns 429 immediately, before it reaches the model.

Note

Your organization's current limits are shown on Org → Overview in the console. These defaults may change. If your workload (for example, a load test) needs higher limits, contact us before you run it.

When you exceed a limit

Exceeding a limit returns 429 Too Many Requests . A requests/min or tokens/min 429 carries x-ratelimit-reset (seconds until the current one-minute window resets). The other headers depend on the limit: a requests/min 429 adds x-ratelimit-limit and x-ratelimit-remaining, and a tokens/min 429 adds x-ratelimit-limit-tokens and x-ratelimit-remaining-tokens. A concurrency 429 is returned by a later stage and carries none of these headers, so back off exponentially and retry.

Retry guidance

Retry 429 and 5xx responses; do not retry 4xx like 400/401/403.
Use exponential backoff with jitter (e.g. 1s, 2s, 4s, ... capped).
On a requests/min or tokens/min 429, wait x-ratelimit-reset seconds instead of guessing a delay. A concurrency 429 has no such header, so back off exponentially.
Cap total retries (e.g. 5) and surface a clear error to the user.

A retry loop that branches on the status and reads x-ratelimit-reset where the response carries it. A concurrency 429 and a 5xx carry no reset header, so those back off exponentially.

Python

import random
import time

def post_with_retry(client, path, payload, max_attempts=5):
    for attempt in range(max_attempts):
        r = client.post(path, json=payload)
        if r.status_code < 400:
            return r
        # 400, 401, 402, 403, 404 and 405 describe the request. Fix it instead.
        if r.status_code != 429 and r.status_code < 500:
            r.raise_for_status()
        reset = r.headers.get("x-ratelimit-reset")
        if reset is not None:
            delay = float(reset)          # requests/min or tokens/min limit
        else:
            delay = min(2 ** attempt, 30) # concurrency 429 and 5xx: no reset header
        time.sleep(delay + random.uniform(0, 1))
    r.raise_for_status()

Credit exhaustion

When your credit balance reaches zero, requests are blocked with 402 Payment Required . Check your balance on the credits page.

For the full list of status codes, see Errors.