Model Routing in 2026: Pay Less on Every LLM Request

TL;DR: Model routing means sending each API request to the cheapest model that can handle it, instead of one frontier model for everything. A rule-based or classifier router sends the easy 70% of traffic to models like GPT-6 Luna ($0.11/$0.55 per 1M tokens on tokens.bd) and reserves Claude Sonnet 5.5 ($2.20/$11.00) for the hard 10% — cutting a modeled 1,000-request/day agent bill from $2,461.80/month to $1,108.59/month (55% off) when combined with a cheap-first cascade. Research backs the shape of these savings: RouteLLM's routers cut benchmark costs up to ~85% on MT-Bench while keeping ~95% of GPT-4's quality, and Ramp cut its own LLM spend ~30% routing 2.75 trillion tokens a month. Don't route latency-critical UIs, >100K-token prompts, or tiny workloads. All prices checked 21 October 2026.

Most teams pick one model — usually the strongest they can afford — and point every request at it. The simple formatting call pays frontier prices. The one-line explanation pays frontier prices. The genuinely hard debugging session pays frontier prices too, and that part is fine, because it needs the intelligence.

The first two don't. That's the waste model routing fixes: one model for everything means paying the top rate for the easiest traffic. On tokens.bd's catalog, the price spread between the cheapest capable models and the frontier tier is roughly 100x per million tokens. Routing is just the discipline of not paying the 100x rate for the 1x problem.

Routing is not fallback (but they pair well)#

A common confusion, worth clearing up in one paragraph: fallback chains decide what happens when a request fails — retry, then a cheaper model, then another provider. Routing decides where a request starts. Fallbacks are about reliability; routing is about cost. The cascade pattern below blends both: a request that the cheap model can't handle well gets escalated to a stronger one, which looks like a fallback but is triggered by quality, not errors. If you run agents in production, you want both layers.

The four routing strategies#

Every router answers one question: what signal is trustworthy enough to pick a model? The strategies differ in how much signal they buy and what they pay for it.

Across all four runs the cascade, also called model tiering: start cheap, and escalate only when a quality check fails, the user asks, or a classifier flags the prompt as hard. Easy requests are guaranteed the cheapest path; hard requests can cost multiple calls. Many production systems combine both — classify up front, cascade as a safety net.

The menu: what the tiers actually cost on tokens.bd#

You can't route without a price ladder. Below are per-million-token prices from tokens.bd's model catalog, checked 21 October 2026 — the same key and base_url for every row, which is what makes per-request routing practical on a gateway.

| Tier | Model | Input / 1M | Output / 1M | | --- | --- | --- | --- | --- | | Cheap | Qwen 3.7 Flash | $0.03 | $0.14 | | Cheap | GPT-6 Luna | $0.11 | $0.55 | | Cheap | Claude Haiku 5.5 | $0.12 | $0.60 | | Cheap | DeepSeek V4.1 Flash 1017 | $0.30 | $0.80 | | Mid | DeepSeek V4 Pro | $0.73 | $2.18 | | Frontier | Claude Sonnet 5.5 | $2.20 | $11.00 | | Frontier | Claude Opus 5.5 | $5.00 | $30.00 | | Frontier | GPT-5.6 Sol | $5.50 | $33.00 | | Frontier | Kimi K3 | $3.30 | $16.50 |

All prices USD per million tokens, checked 21 October 2026 on tokens.bd/models. The catalog holds 60+ models from 10 providers.

One row does most of the persuading: a routine request — say 4K input tokens, 1K output — costs $0.0198 on Sonnet 5.5 and $0.00099 on GPT-6 Luna. Same wire format, same API shape, 20× cheaper. Multiply that by the share of your traffic that's routine, and you have the entire business case.

For a fast new release worth comparing alongside the cheap tier, see Mercury 3: The 1,068 tok/s Diffusion LLM Explained.

The worked math: 1,000 agent requests a day#

Assumptions, stated so you can replace them: an agent making 1,000 requests/day. 70% routine (4K in / 1K out), 20% standard coding work (30K in / 5K out), 10% hard reasoning (100K in / 20K out). Prices from the table above.

Plan A — everything on Claude Sonnet 5.5:

| Traffic | Per-request cost | × requests | Per day | | --- | --- | --- | --- | --- | | 700 routine | 0.004×$2.20 + 0.001×$11.00 = $0.0198 | ×700 | $13.86 | | 200 standard | 0.03×$2.20 + 0.005×$11.00 = $0.121 | ×200 | $24.20 | | 100 hard | 0.1×$2.20 + 0.02×$11.00 = $0.44 | ×100 | $44.00 | | Total | | | $82.06/day → $2,461.80/month (≈৳304,032) |

Plan B — prompt routing (classify up front): routine → GPT-6 Luna, standard → DeepSeek V4.1 Flash 1017, hard → Sonnet 5.5.

| Traffic | Per-request cost | × requests | Per day | | --- | --- | --- | --- | --- | | 700 routine → GPT-6 Luna | 0.004×$0.11 + 0.001×$0.55 = $0.00099 | ×700 | $0.693 | | 200 standard → DeepSeek V4.1 Flash | 0.03×$0.30 + 0.005×$0.80 = $0.013 | ×200 | $2.60 | | 100 hard → Sonnet 5.5 | $0.44 | ×100 | $44.00 | | Total | | | $47.29/day → $1,418.79/month (≈৳175,221) — 42.4% cheaper |

Plan C — routing + cheap-first cascade: same as B, but the hard tier tries DeepSeek V4 Pro first ($0.73/$2.18) and escalates to Sonnet only when the verifier isn't confident. Assume half escalate — pessimistic on purpose, since escalations re-process the full input at the frontier price.

| Traffic | Per day | | --- | --- | --- | | Routine + standard (as in Plan B) | $3.29 | | 100 hard attempts on DeepSeek V4 Pro: 0.1×$0.73 + 0.02×$2.18 = $0.1166 each | $11.66 | | 50 escalations re-processed on Sonnet 5.5 at $0.44 each | $22.00 | | Total | $36.95/day → $1,108.59/month (≈৳136,911) — 55.0% cheaper |

USD→BDT at ≈৳123.5 per $1 (mid-market, 21 October 2026). Monthly = 30 days.

Two honest caveats. First, the hard tier still dominates the bill in every plan — routing shrinks the 90% of traffic that was never the problem and leaves the expensive 10% expensive. That's the point: it stops you from also paying frontier rates for the easy 90%. Second, the cascade's savings depend on the escalation rate; if 90% of your "hard" traffic escalates, the cascade is just a latency tax. Measure your escalation rate before committing.

The verifier is the whole game#

A cascade without a verifier is just a slow way to call the expensive model. Three verifier designs dominate in 2026, in order of cost:

1. Structured-output validation (cheapest). If you asked for JSON and the cheap model's output doesn't parse, escalate. Free, deterministic, and catches the most common cheap-model failure mode on tool-using agents. (Our structured outputs guide covers the schemas.)
2. Self-consistency. Ask the cheap model twice; escalate on disagreement. Doubles the cheap call, which is still far below one frontier call.
3. Judge model. A small dedicated model scores the answer's confidence. Adds 300–800 ms, but catches subtle wrongness the other two miss.

The escalation path should log which model actually served each request — silent escalation hides degradation exactly the way silent fallbacks do (see the fallback post's rule #2). And set per-rung timeouts: a request that escalates twice pays the latency of cheap + verifier + strong, typically 600–900 ms extra on the tail — fine for background agents, unacceptable for interactive chat.

A rule-based router in twenty lines#

The cheapest router has no model in it at all — just a route map. This works with any OpenAI-compatible client, including tokens.bd's (one base_url, one key, 60+ models behind it):

import re
from openai import OpenAI

client = OpenAI(base_url="https://tokens.bd/v1", api_key="tkbd_...")

ROUTES = [
    # (pattern, model, reason)
    (re.compile(r"^(summar|classif|extract|format|translate)\b", re.I),
     "openai/gpt-6-luna", "routine transform"),
    (re.compile(r"\b(debug|refactor|architect|optimiz|race condition)\b", re.I),
     "anthropic/claude-sonnet-5-5", "hard reasoning"),
]
DEFAULT = "deepseek-ai/deepseek-v4.1-flash-1017"  # mid-tier catch-all

def chat(prompt: str, **kwargs):
    for pattern, model, reason in ROUTES:
        if pattern.search(prompt):
            chosen, why = model, f"rule:{reason}"
            break
    else:
        chosen, why = DEFAULT, "rule:default"
    resp = client.chat.completions.create(model=chosen, messages=[{"role": "user", "content": prompt}], **kwargs)
    resp.model_served = (chosen, why)  # log this; "served by" is the column
    return resp

Start here — rules cover a surprising share of production traffic for $0 of added cost — and graduate to semantic or classifier routing only when your logs show the rules misrouting often enough to matter. When the router itself is an LLM call, its cost is the classifier's input tokens: classifying a 4K-token prompt on GPT-6 Luna costs about $0.00044 — under 1% of a single hard-tier call.

When NOT to route#

Routing is a cost optimization with real tradeoffs. Skip it when:

• Latency is the product. Cascades add hundreds of milliseconds on the escalating tail; classifier routers add it on every request. Interactive chat with an SLA usually wants one fast model.
• Prompts are huge. Escalation re-processes the entire input at the second model's price. Above ~100K tokens of context, one call to the right model beats two calls to the cheap one.
• Every request is accuracy-critical. Medical, legal, financial review — the savings don't justify even a small quality regression. Pay the frontier rate deliberately.
• Volume is small. If your monthly bill is $40, the engineering time to build and monitor a router costs more than routing saves. Come back when the bill hurts.

Routing × caching: the multiplier#

Routing and prompt caching multiply rather than add, because caches are per model: routing keeps the same request types on the same models, so cache hit rates climb instead of fragmenting across tiers. Keep the 70% routine tier on one cheap model with a stable system prompt, and cache reads — typically 90–99% cheaper than fresh input — do the rest. And once routing is in, put the ceiling from the spend-caps pattern on each key: a misbehaving router that escalates everything is just an expensive way to call Opus.

FAQ#

How much does model routing actually save? It depends on your traffic mix, but the shape is consistent: research and production reports cluster at 30–60% savings. RouteLLM's trained routers cut costs up to ~85% on MT-Bench (45% on MMLU, 35% on GSM8K) while keeping ~95% of GPT-4's quality; Ramp cut ~30% routing 2.75 trillion tokens a month; FrugalGPT's cascades reported up to 98% in their experiments. In the worked example above — 1,000 agent requests/day on tokens.bd prices — prompt routing saved 42.4% and routing plus a cascade saved 55.0%.

What's the difference between routing and fallback chains? Routing picks where a request starts, based on cost and difficulty. Fallback chains decide what happens when a request fails, based on errors and outages. A cascade blends them: escalation on low quality looks like a fallback but is triggered by the verifier, not by an error code.

Does routing add latency? Rule-based and cost-aware routing add effectively none. Semantic routing adds 50–150 ms for the embedding lookup; LLM-classifier routing adds 300–800 ms for the classifier call. Cascades add latency only on the escalating tail — typically 600–900 ms extra when a request climbs two rungs. That's why cascades suit background agents better than interactive chat.

Won't a cheap model just give worse answers? On hard reasoning, yes — which is why the hard tier stays on the frontier model. The empirical finding behind routing research is that on routine requests (classification, extraction, formatting, simple Q&A), cheap models match frontier quality, so the 70% of traffic that's routine loses nothing. The verifier exists to catch the requests that were misclassified.

Where do I start? With the rule-based router above and the tokens.bd model catalog: pick one cheap tier, one frontier tier, log which model serves each request, and measure for a week. Add a verifier and a mid tier only when the logs justify them. The monthly bill estimation method gives you the before-number to beat, and the OpenRouter comparison covers how gateway failover interacts with your routing.

All prices USD per million tokens, checked 21 October 2026 on tokens.bd/models. USD→BDT at ≈৳123.5. Benchmark and production figures are reported by their sources (linked below); provider list prices change — the maker's page is the authority, and the catalog for prices on Tokens.

Sources: tokens.bd model catalog · Codingscape — LLM model routing: Uber, AT&T, and Ramp cut AI costs 30–56% · LLMDeal — Does model routing actually cut your LLM bill? · Cluster, Route, Escalate — cost-aware LLM cascade paper · Solangle — LLM Model Routing in 2026: Cascade Cuts API Costs by 60% · Techsy — LLM Router: Cut Costs 60% With Model Routing · dev.to — Model Routing 101 · dev.to — How Semantic Routing Cut My LLM Costs by 70% · Swfte — Mixture-of-Routers and 2026 LLM Routing Techniques · Valuta.EX — USD to BDT rate

Routing decisions themselves can be decision calls — see OpenAI Decisions API vs Jev: Decisions Without the Essay.

For computer-use work, GPT-6.1 Terra: $1 Computer Use, Explained is a strong mid-tier rung to route routine screen work to before escalating to Sol.