Four separate limits can stop a request before it reaches a model: requests per minute, concurrent requests, plan usage windows, and a key's monthly spend cap. Token counting has a limit of its own. Upstream providers have their own rate limits on top. This page explains each one, the error it returns, and how a client should react.
Rate limits at a glance#
| Limit | Scope | Default | Error | Retry-After |
|---|---|---|---|---|
| Requests per minute | Each person (all their keys) | 60 RPM, or your plan's value | 429 rate_limited | Seconds until the next minute |
| Concurrent requests | Account (a team shares it) | Plan's limit; 10 with a plan, 3 without | 429 concurrency_limit | 2 |
| Usage windows | Subscription | Set by the plan | 429 window_exhausted | Seconds until the window resets |
| Monthly spend cap | One key | None unless set at key creation | 403 monthly_spend_cap_exceeded | Not sent |
| Token counting | One key | 600 per minute | 429 rate_limited | Seconds until that key's minute ends |
| Upstream provider limit | Provider | Set by the provider | 429 rate_limit_exceeded | Passed through if the provider sent one |
Your plan's exact numbers are shown in billing and explained in plans and wallet.
Requests per minute#
The per-minute limit counts each person's requests in fixed one-minute buckets that start on the clock minute. Every key a person uses shares the same bucket, so creating more keys doesn't raise it. On a team, each member has their own minute at the team plan's rate. The dashboard playground has its own separate limit of 10 RPM and doesn't use up your API allowance.
When you go over, you get 429 rate_limited with Retry-After set to the seconds remaining in the current minute, so the wait is never more than 60 seconds.
Requests rejected for insufficient funds (402 insufficient_credits or no_funding) or an exhausted usage window still count toward the minute; requests rejected by the concurrency limit don't. A client that retries a 402 in a tight loop will also hit the per-minute limit. Don't retry 402s at all.
GET /v1/models, GET /v1/tokens/usage and POST /v1/messages/count_tokens don't count. Token counting has its own limit instead: 600 requests a minute per key, answered with 429 rate_limited and a Retry-After when a key goes over. See token counting.
Concurrency limit#
Concurrency is the number of requests in flight at once on your account; on a team, all members share it. A streaming request occupies a slot from admission until the stream finishes, so a coding agent that opens several streams in parallel, or a script that fans out with asyncio.gather, can hit this before the per-minute limit.
The 429 concurrency_limit response carries Retry-After: 2. The fix is usually to cap parallelism on your side, for example with a semaphore sized below your plan's limit, rather than retrying harder. Close or abort streams you no longer need; an abandoned stream still holds its slot until it ends.
Usage windows#
Subscription plans can define usage windows, each limited in credits (expressed in USD) or in requests:
| Window | type in the API | Resets |
|---|---|---|
| 5-hour session | session_5h | 5 hours after the request that opened the window |
| Weekly | weekly | Every 7 days from the start of your subscription period; the last week ends with the period |
| Monthly | monthly | At the end of the subscription period |
When any window is used up, requests return 429 window_exhausted and Retry-After is the number of seconds until that window resets. That can be hours, so don't retry automatically: show the reset time to the user, or stop the job. Each admitted response says how much of the tightest window is left (x-tokens-window-remaining, see the headers). Check the remaining amounts before starting a long agent run:
curl -s https://tokens.bd/v1/tokens/usage -H "Authorization: Bearer $TOKENS_API_KEY"To keep a long run going past a window, set What pays to Auto in the dashboard: once a window is used up, your wallet pays until it resets, and wallet-paid requests don't count toward the windows. See choose what pays. The per-minute and concurrency limits still apply.
The response shape is documented in models and usage. Alerts at 50, 75, 90 and 100 percent are covered in usage and alerts.
Monthly spend caps per key#
A key can have a monthly spend cap in USD, set when you create it. Spend is counted per calendar month (UTC). Before each request the gateway checks the key's spend so far plus the worst-case cost of the new request, based on its max_tokens (or 8,192 output tokens if unset). Near the cap, a request with a large max_tokens can be refused while a smaller one would pass.
The error is 403 monthly_spend_cap_exceeded, not a 429, because waiting a few seconds won't help. Caps can't be edited after creation; if you need a higher one, create a new key. If you only change one setting on a key that goes into a shared CI system or an agent you don't watch, make it the spend cap. More in API keys.
Retry-After and rate limit headers#
Every admitted request says what is left, so a client or an agent's usage monitor can slow down before it is refused:
| Header | Value |
|---|---|
x-ratelimit-limit-requests | Your requests-per-minute limit |
x-ratelimit-remaining-requests | Requests left in this minute, after this one |
x-ratelimit-reset-requests | Time until the minute's count starts again, such as 23s |
x-tokens-window-type | The usage window with the least left: session_5h, weekly or monthly |
x-tokens-window-unit | What it counts: usd or requests |
x-tokens-window-limit, x-tokens-window-remaining | Its size and what is left of it, in that unit |
x-tokens-window-reset | When it resets, as an ISO 8601 time in UTC |
x-tokens-credits-remaining | Plan credits available now, in USD |
x-tokens-wallet-remaining | Wallet available now, in USD (balance plus any sign-up bonus) |
The figures are the ones billing shows at the moment the request was admitted, for the account the key bills (a team key shows the team's): what is available after the requests still running, this one included. A request is charged its real cost when it ends, so the next response usually shows a little more used. A usage window's used grows when a request ends, not when it starts.
A header is left out when it doesn't apply:
- No
x-tokens-window-*without a plan, on Wallet only, or while no window has started yet (nothing used since the last reset). - No
x-tokens-credits-remainingon Wallet only, once the wallet pays past a reached limit or a model's cap, or for a model your plan makes free or covers with a token allowance. - No money header for a model your plan makes free, a token allowance, or a request a model's free access paid for (that one carries
x-tokens-free-requests-limitandx-tokens-free-requests-remaininginstead).
The 429 rate_limited error carries the three x-ratelimit-* headers (with 0 remaining) next to Retry-After. Other errors carry only Retry-After, in seconds, when waiting helps. There are no token-per-minute headers (x-ratelimit-*-tokens): the gateway doesn't limit tokens per minute. Browser code can read all of these headers. To see what is left without sending a request, poll GET /v1/tokens/usage. Every admitted request also says what paid for it in x-tokens-paid-by. All the headers are listed in errors.
Client-side backoff example#
The OpenAI and Anthropic SDKs retry some 429 and 5xx errors on their own (max_retries, 2 by default). The example below turns that off and handles it explicitly, so window_exhausted isn't retried and Retry-After is honored.
import os
import random
import time
import openai
from openai import OpenAI
client = OpenAI(
base_url="https://tokens.bd/v1",
api_key=os.environ["TOKENS_API_KEY"],
max_retries=0,
)
RETRYABLE = {429, 500, 502, 503, 504}
def create_with_backoff(max_attempts: int = 5, **kwargs):
delay = 1.0
for attempt in range(1, max_attempts + 1):
try:
return client.chat.completions.create(**kwargs)
except openai.APIStatusError as e:
if (
e.status_code not in RETRYABLE
or e.code == "window_exhausted"
or attempt == max_attempts
):
raise
try:
wait = float(e.response.headers.get("retry-after", delay))
except ValueError:
wait = delay
request_id = e.response.headers.get("x-tokens-request-id")
print(f"{e.status_code} {e.code} (request {request_id}), retrying in {wait:.1f}s")
except openai.APIConnectionError:
if attempt == max_attempts:
raise
wait = delay
time.sleep(min(wait, 60) + random.uniform(0, 0.5 * delay))
delay = min(delay * 2, 30)
resp = create_with_backoff(
model="deepseek/deepseek-v4.1-flash",
messages=[{"role": "user", "content": "Say hello."}],
max_tokens=50,
)
print(resp.choices[0].message.content)import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://tokens.bd/v1",
apiKey: process.env.TOKENS_API_KEY,
maxRetries: 0,
});
const RETRYABLE = new Set([429, 500, 502, 503, 504]);
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
function retryAfterMs(err: InstanceType<typeof OpenAI.APIError>): number | undefined {
const h: unknown = err.headers; // a Headers object or a plain record, depending on SDK version
const value =
h instanceof Headers
? h.get("retry-after")
: (h as Record<string, string | undefined> | undefined)?.["retry-after"];
return value ? Number(value) * 1000 : undefined;
}
async function createWithBackoff(
body: OpenAI.Chat.ChatCompletionCreateParamsNonStreaming,
maxAttempts = 5
) {
let delay = 1000;
for (let attempt = 1; ; attempt++) {
try {
return await client.chat.completions.create(body);
} catch (err) {
if (!(err instanceof OpenAI.APIError) || attempt >= maxAttempts) throw err;
const connectionError = err instanceof OpenAI.APIConnectionError;
if (
!connectionError &&
(!RETRYABLE.has(err.status ?? 0) || err.code === "window_exhausted")
) {
throw err;
}
const wait = (!connectionError && retryAfterMs(err)) || delay;
await sleep(Math.min(wait, 60_000) + Math.random() * 0.5 * delay);
delay = Math.min(delay * 2, 30_000);
}
}
}
const resp = await createWithBackoff({
model: "deepseek/deepseek-v4.1-flash",
messages: [{ role: "user", content: "Say hello." }],
max_tokens: 50,
});
console.log(resp.choices[0].message.content);Every successful retry is a billed request, so keep the attempt count low. The full list of retryable and non-retryable codes is in errors.