API Rate Limits Explained: RPM, TPM, and 429 Fixes (2026)

TL;DR: Every LLM API caps how fast you can call it, and every provider measures "fast" differently: OpenAI meters requests and tokens per minute by spend tier, Anthropic splits tokens into input and output per minute, Gemini adds a daily request cap on its free tier, and DeepSeek counts concurrent requests instead of per-minute quotas. On 19 October 2026, an entry-level paid account gets you 5,000 RPM on OpenAI, 50 RPM on Anthropic, or 10 RPM on Gemini's free tier — a 100x spread that decides your architecture before you write a line of retry code.

Your app works in testing. It works in the demo. Then real traffic arrives and every response is 429 Too Many Requests. This is the most common production failure in LLM-powered apps, and it is almost never about your code being wrong — it is about a throughput ceiling you never designed for. This post explains the three ceilings (RPM, TPM, concurrency), what each major provider actually gives you as of 19 October 2026, and the retry patterns that keep 429s from becoming outages.

The three ceilings#

Providers meter throughput on up to three independent axes. Hitting any one of them returns a 429 — you don't need to exceed all three.

A swarm of tiny requests trips RPM. A few giant-context requests trip TPM. A Promise.all over hundreds of calls trips concurrency. The fix depends on which ceiling you hit — so the first debugging step is always reading the error body and headers, not adding a sleep.

What each provider gives you (checked 19 October 2026)#

Entry-level paid (or free, where that's the story) ceilings. Providers change these quietly; your project's live dashboard is the final word.

How to read this table:

• OpenAI rebuilt its tiers on 6 October 2026 into three — Build, Launch, Grow — qualifying on cumulative credit purchases ($5 / $100 / $500). The small model gets a much wider lane than the flagships: at Grow, Luna allows 30,000 RPM and 180M TPM against 15,000 RPM and 40M TPM for Astra/Sol/Terra. If the job can be done by the small model, the limits quietly say so.
• Anthropic splits tokens into input TPM and output TPM — a combined TPM number is misleading. Cached input tokens generally don't count toward input TPM on current Claude models, so a cache-heavy workload has a higher effective limit than the table suggests. Limits are per model family, and the token-counting endpoint is free with its own 5,000 RPM allowance on the Start tier — count before you send.
• Gemini's free tier varies wildly by model: 2.5 Flash gets 1,500 requests/day while 2.5 Pro gets 50/day and 5 RPM. Limits apply per Google Cloud project, not per API key. Paid tiers unlock on spend plus waiting days ($100 + 3 days for Tier 2, $1,000 + 30 days for Tier 3). Note the free-tier condition: Google may use free-tier inputs to train models; paid tiers don't.
• DeepSeek doesn't publish RPM or TPM at all — it meters concurrent requests per account (all your keys share one pool; making more keys doesn't help). Above the ceiling you get a 429. Its Search/Fetch APIs are separately capped at 5 requests/second per account.

Not every 429 is a rate limit#

The status code is the same; the cause isn't. Before you tune your retry logic, check which of these you actually hit:

1. Throughput 429 — you exceeded RPM/TPM/concurrency. Wait and retry; the provider usually tells you how long via a Retry-After header.
2. Billing 429 — insufficient_quota on OpenAI means the money ran out, not the minute. Retrying is pointless; top up or fix auto-recharge.
3. Window 429 — plan-level usage windows (tokens.bd returns 429 window_exhausted when a 5-hour, weekly, or monthly window is spent). Sleeping for hours inside a request loop is rarely what you want — surface it to the user instead.
4. Upstream 429 — on a gateway, the provider behind it is the one that's full. tokens.bd retries a provider 429 automatically on another source for the same model and only surfaces rate_limit_exceeded when no source is left.

On tokens.bd specifically, the error code tells you exactly which ceiling: rate_limited (60 RPM per user), concurrency_limit (too many in flight), model_limit_reached (the model's billing-period allowance), or window_exhausted (usage window). Each carries a Retry-After header with the real wait — seconds for rate limits, up to hours or days for windows.

Which ceiling bites first: three worked examples#

These use the entry-tier numbers above (checked 19 October 2026). Replace with your tier's figures.

Example 1 — the agent swarm (Anthropic Build). You fan out 50 parallel requests, each with 2,000 input tokens: 100,000 input tokens total. RPM ceiling is 50 — the requests just fit in one minute. Input TPM ceiling is 40,000 — the tokens need 100,000 / 40,000 = 2.5 minutes to clear. This burst is TPM-bound, not RPM-bound: spreading the same 50 requests over 3 minutes fixes it without touching concurrency.

Example 2 — the long-prompt pipeline (OpenAI Launch, Luna). 10,000 RPM and 10,000,000 TPM. The breakeven is 10,000,000 / 10,000 = 1,000 tokens per request: requests averaging under 1,000 tokens hit the RPM wall first; requests averaging over 1,000 hit TPM first. Most coding-agent requests (large context, moderate count) are TPM-bound on OpenAI — which is exactly why Luna's 2–10x TPM lane over flagships matters.

Example 3 — the free-tier side project (Gemini 2.5 Flash). 1,500 requests/day. A cron hitting the API every 30 seconds makes 2,880 requests/day — nearly double the cap, and it goes dark by mid-afternoon. The fix isn't code; it's either the paid tier or a cheaper-per-request design.

The pattern: figure out your requests-per-minute and tokens-per-minute at peak, divide by each ceiling, and the smallest headroom is your binding constraint. Design for that one.

The retry patterns that actually work#

1. Honor Retry-After before anything else. It's the provider telling you exactly how long to wait. Blind fixed sleeps either wait too long or retry too early.

2. Exponential backoff with jitter. When there's no Retry-After, wait 1s, 2s, 4s… up to a cap, and add random jitter so a fleet of clients doesn't retry in lockstep and re-trigger the limit. Cap retries at ~5 attempts for throughput 429s.

3. Never retry 400s or 401s. A malformed request or a bad key fails identically every time — retrying burns quota and logs. Only retry 429s, timeouts, and 5xx errors.

4. Bound your concurrency client-side. A semaphore around your request pool, sized under your concurrency ceiling, prevents the burst from ever forming. This is the single highest-leverage change for agent swarms.

5. Read the rate-limit headers. OpenAI returns x-ratelimit-remaining-requests / x-ratelimit-remaining-tokens; Anthropic returns anthropic-ratelimit-*-limit and *-remaining. Proactive throttling on remaining budget beats reactive 429 handling. (tokens.bd doesn't send X-RateLimit-* headers — use Retry-After on its 429s instead.)

import time, random
from openai import OpenAI, RateLimitError, APIError

client = OpenAI(base_url="https://tokens.bd/v1")  # any OpenAI-compatible endpoint

def call_with_backoff(**kwargs):
    for attempt in range(5):
        try:
            return client.chat.completions.create(**kwargs)
        except RateLimitError as e:
            # Prefer the server's Retry-After when present
            retry_after = getattr(e, "retry_after", None)
            delay = float(retry_after) if retry_after else (2 ** attempt) + random.uniform(0, 1)
            time.sleep(min(delay, 60))
        except APIError as e:
            if e.status_code and e.status_code >= 500 and attempt < 4:
                time.sleep((2 ** attempt) + random.uniform(0, 1))
                continue
            raise
    raise RuntimeError("rate limit persisted after retries")

import asyncio

# Concurrency guard: never exceed your plan's in-flight ceiling
semaphore = asyncio.Semaphore(50)

async def bounded_call(coro_fn, *args):
    async with semaphore:
        return await coro_fn(*args)

How a gateway changes the rate-limit game#

When you call providers directly, you juggle four different limit systems, four tier ladders, and four header dialects. A gateway like tokens.bd collapses that to one:

• One ceiling to learn. 60 requests/minute per user and a plan-set concurrency limit (3 without a plan), on your account — not per key, so key sprawl can't accidentally multiply your footprint.
• Automatic failover on 429s. If the upstream source for a model answers 429 (or 502/503/504, or drops the connection), the request is retried on another source for the same model before you ever see an error. You only get rate_limit_exceeded when every source is full.
• Usage windows instead of surprise bills. Plans spread allowance over a rolling 5-hour session plus weekly and monthly ceilings, with email alerts at 50%, 75%, and 90% — and per-key monthly spend caps so one runaway agent can't burn the budget. A 429 with a Retry-After is recoverable; an empty wallet at 3 AM is a different kind of outage.
• One dashboard for all of it. GET /v1/tokens/usage (free, unmetered) returns your windows, wallet balance, and the calling key's limits — poll it from scripts or CI instead of guessing.

The broader lesson stands regardless of gateway: rate limits are a capacity-planning input, not an error to be surprised by. Know your peak RPM and TPM, know which ceiling binds, and put the backoff and concurrency guard in on day one. For the cost side of the same equation — what each of those tokens bills — see how your LLM API bill really works. And if you're comparing providers on price rather than throughput, the OpenRouter alternative guide covers the fee math.

FAQ#

What's the difference between a 429 rate-limit error and a 429 insufficient-quota error? The HTTP status is identical; the error body tells them apart. A rate-limit 429 means "too fast — wait and retry," usually with a Retry-After header. An insufficient-quota/billing 429 means "no credit left" — retrying changes nothing; top up the balance or enable auto-recharge. Always log the response body, not just the status code.

Will creating more API keys raise my rate limit? No — on every major provider. Anthropic limits apply per organization, DeepSeek's concurrency pool is account-level across all keys, Gemini's quotas are per Google Cloud project, and tokens.bd's limits apply per account, not per key. More keys just give you more strings that share one bucket.

Do cached tokens count against rate limits? On Anthropic, cached input tokens generally don't count toward input TPM on current Claude models — prompt caching effectively raises your throughput ceiling as well as cutting cost (see prompt caching explained). OpenAI counts cached tokens in its own way per model; check your tier's limits page rather than assuming.

How should I handle 429s from a coding agent? Don't let the agent hammer retries itself — route its calls through a wrapper that honors Retry-After, backs off exponentially with jitter, and caps in-flight requests under your concurrency limit. On tokens.bd, set a per-key monthly spend cap and turn on the 50/75/90% usage alerts before any long unattended run (usage and alerts docs).

Why did my app work in testing but 429 in production? Testing never generates production-shaped traffic: one user versus hundreds, sequential calls versus parallel fan-out, small prompts versus full repo context. Multiply your test traffic by your expected concurrency and compare against each ceiling in the table above — the gap is usually obvious once you do the division. Browse the full model list and prices on the tokens.bd model catalog when choosing which model your peak traffic should run on.

Sources#

Rate-limit figures checked 19 October 2026 against sources dated within the last two weeks where possible:

• OpenAI tier rebuild (Build/Launch/Grow, 6 Oct 2026): https://cellcog.ai/blog/openai-api-usage-tiers/ · https://www.scriptbyai.com/rate-limits-openai-api/ · https://intuitionlabs.ai/articles/chatgpt-api-pricing-2026-token-costs-limits · https://folkfox.com/openai-api-rate-limits-three-tiers-decisions-api/
• Anthropic three-axis limits and tiers: https://llmapi.pro/blog/anthropic-api-rate-limits-usage-tiers · https://callsphere.ai/blog/claude-api-rate-limits-best-practices · https://aikitguides.com/blog/openai-rate-limits-guide/
• Gemini free/paid tiers: https://ampm-aiops.com/en/guides/google-ai-studio-review-2026/ · https://www.anygen.io/showcase/gemini-api/index.html (citing Google AI for Developers docs, updated 18 Aug 2026)
• DeepSeek concurrency limits (checked 5 Oct 2026): https://chat-deep.ai/docs/api-rate-limits/ · https://deepseek-usa.ai/docs/api/
• tokens.bd first-party docs (verified live 19 Oct 2026): https://tokens.bd/docs/usage-and-alerts · https://tokens.bd/docs/overview