TL;DR: OpenAI's prompt caching is fully automatic — no code changes, no markers, no write fee on current models. Prompts of 1,024+ tokens get cached in 128-token increments, and the discount depends on the model: 50% on GPT-4o, 75% on GPT-4.1 and o3, 90% on the GPT-5 family, 95% on GPT-6.1 Sol. A 50K-token agent context on GPT-5.5 drops from $25.00 to ~$2.73 per 100 requests (~89% off). All prices checked 16 October 2026.
Every OpenAI API call re-processes your system prompt, tool definitions, and conversation history from scratch — unless prompt caching kicks in. When it does, the repeated prefix is billed at a steep discount instead of full price. The unusual part: unlike Anthropic's version, you don't have to do anything to get it. There are no cache_control markers, no SDK changes, no configuration. If your prompt is long enough and stable enough, the discount just happens.
This post is the OpenAI-specific deep dive: exactly how the automatic caching works, what each model family actually saves you, how to verify it's working on your traffic, and the worked math. For the general mechanics, see Prompt Caching Explained; for the "should I cache or just use a cheaper model" question, see Prompt Caching vs Model Downgrade; for the three-provider comparison, see Prompt Caching Compared: Anthropic vs OpenAI vs Bedrock.
How OpenAI's caching actually works#
OpenAI's implementation is prefix matching on the provider's side. When a request arrives, OpenAI walks the prompt from the longest prefix backward, looking for a previously processed prefix on the same machine. On a hit, the saved key-value tensors are reused instead of recomputed; only the new tail of the prompt is processed at full price.
The rules that decide whether you qualify, from OpenAI's current documentation:
- Automatic, on by default. Enabled for supported models with no opt-in. There is a Prompt Caching Dashboard to monitor hit rates.
- Minimum 1,024 tokens. The prompt prefix must meet the model's minimum cacheable length — 1,024 tokens for GPT-5.6 and later (it varies by request settings on earlier models). Under that,
cached_tokensis always zero. Hidden OpenAI system tokens don't count toward the minimum. - 128-token increments. Reported
cached_tokensis the last matched breakpoint minus hidden system tokens, rounded down to the nearest multiple of 128. - Exact prefix match. The entire rendered prefix must match byte-for-byte: developer messages, tool definitions (names, descriptions, schemas, ordering), conversation history, images, structured-output schemas. Change anything before the breakpoint and everything after it stops matching. Notably,
reasoning.effortandtext.verbositychanges can also break the prefix. - No write fee (on current models). The first request processes the prefix at the standard input rate; repeats get the discount. There is no Anthropic-style write premium — with one new exception covered below.
What can be cached: the complete messages array (system/developer/user/assistant), images in user messages, tool definitions, and structured-output schemas.
The discount is model-dependent — and that's the whole story#
This is the number most people get wrong. "OpenAI gives 50% off cached tokens" was true in the GPT-4o era and is now wrong for newer models. The discount tiers, checked 16 October 2026:
| Model family | Discount on cached input | Example: input → cached (per 1M) |
|---|---|---|
| GPT-6.1 Sol | 95% | $2.00 → $0.10 |
| GPT-5 family (5.5, 5.4, 5.x, mini, nano) | 90% | GPT-5.5: $5.00 → $0.50 |
| GPT-4.1 family | 75% | GPT-4.1: $2.00 → $0.50 |
| o3 / o4-mini | 75% | o3: $2.00 → $0.50 |
| GPT-4o / o-series | 50% | GPT-4o: $2.50 → $1.25 |
Discount tiers and example rates checked 16 October 2026 against OpenAI's pricing and current cost analyses.
Two things follow from this table. First, the "cache discount ratio" — input divided by cached — is what decides how much a warm cache changes your bill: 20× on GPT-6.1 Sol, 10× on GPT-5, 4× on GPT-4.1, 2× on GPT-4o. Second, GPT-4o's deprecation matters here beyond the migration headline: GPT-4o was deprecated on 14 October 2026 with API shutdown on 14 January 2027, and moving that workload to a GPT-5-class model roughly doubles the caching discount while the base input price stays comparable.
The GPT-5.6 wrinkle: the free lunch is ending on the newest models#
One important update from OpenAI's current docs: on GPT-5.6 and later, cache writes now cost 1.25× the standard input rate, with reads at 0.1× (0.05× on GPT-6.1 Sol). The same docs introduce explicit cache breakpoints (prompt_cache_breakpoint on content blocks, up to four cache writes per request) and a prompt_cache_options.ttl setting (default 30 minutes).
In other words, OpenAI is converging toward Anthropic's model on its newest generation: you can now pay to write and choose where breakpoints land. On everything before GPT-5.6, the old deal holds — automatic, implicit, no write charge. If you're on GPT-5.5 or earlier, nothing changes for you today; if you're evaluating GPT-5.6+, budget the 1.25× write into your break-even math the same way Anthropic users do.
How to see it working on your traffic#
Every API response tells you exactly what got cached. On Chat Completions, read usage.prompt_tokens_details.cached_tokens; on the Responses API, usage.input_tokens_details.cached_tokens (plus cache_write_tokens on newer models). A minimal check:
from openai import OpenAI
client = OpenAI()
resp = client.chat.completions.create(
model="gpt-5.5",
messages=[
{"role": "system", "content": "...your long stable prompt..."},
{"role": "user", "content": "Summarize the key risks."},
],
)
details = resp.usage.prompt_tokens_details
print(f"input: {resp.usage.prompt_tokens}, cached: {details.cached_tokens}")
# Second identical-prefix call: cached_tokens jumps, bill drops.If cached_tokens stays zero across repeated calls, your prefix isn't stable — the usual culprits are timestamps or request IDs in the system prompt, non-deterministic tool JSON serialization, or a prompt under 1,024 tokens. The failure is silent: no error, just full-price input.
Routing, keys, and TTLs: the three knobs that control your hit rate#
Caching happens per machine, and OpenAI routes your request by hashing the initial tokens (after hidden system content) plus an optional routing hint. Three settings matter:
prompt_cache_key. A string you supply to keep related requests on the same machines. On models before GPT-5.6, use one stable key per workload (e.g. per customer segment) — OpenAI suggests aiming for about 15 requests per minute per key; above that, overflow spills to machines with cold caches. On GPT-5.6+, routing is automatic and the key is mainly for per-customer usage accounting.- Retention tiers.
in_memory(the default for zero-data-retention orgs): entries stay active through roughly 5–10 minutes of inactivity, up to one hour.24h: extended retention, typically ~30 minutes and up to 24 hours — but it offloads tensors to storage, so it is not the no-data-retention path. Reusing a prefix refreshes its lifetime without another write charge. - Prefix discipline. Stable content first, volatile content last. Put timestamps, user IDs, and per-request data at the end of the prompt, never in the system block. Keep tool definitions byte-identical between calls.
Worked example: the same agent workload on three model families#
Assumptions, stated so you can rerun them: a coding agent with a 50,000-token stable context, 100 requests in one cache lifetime, every request after the first hitting the warm cache. Output tokens excluded — caching never discounts output. Prices checked 16 October 2026; pre-GPT-5.6 behavior (no write fee).
| Line item | GPT-5.5 ($5.00 → $0.50) | GPT-4.1 ($2.00 → $0.50) | GPT-4o ($2.50 → $1.25) |
|---|---|---|---|
| Without caching | 100 × 0.05M × $5.00 = $25.00 | 100 × 0.05M × $2.00 = $10.00 | 100 × 0.05M × $2.50 = $12.50 |
| First request (full price) | 0.05M × $5.00 = $0.25 | 0.05M × $2.00 = $0.10 | 0.05M × $2.50 = $0.125 |
| 99 cache reads | 99 × 0.05M × $0.50 = $2.475 | 99 × 0.05M × $0.50 = $2.475 | 99 × 0.05M × $1.25 = $6.1875 |
| With caching | $2.73 | $2.58 | $6.31 |
| Saving | $22.27 (~89%) | $7.43 (~74%) | $6.19 (~49.5%) |
Arithmetic check: $25.00 − $2.73 = $22.27; $10.00 − $2.58 = $7.43; $12.50 − $6.31 = $6.19. In taka at today's rate (1 USD ≈ ৳123.05, Finnhub 16 October 2026): the GPT-5.5 workload drops from ~৳3,076 to ~৳335 per 100 requests — about ৳2,741 saved per day of agent traffic.
The pattern is worth internalizing: the same warm cache produces wildly different savings depending on the model's tier. This is also why the "migrate off GPT-4o" advice keeps paying twice — you get the newer model's price and a deeper caching discount.
To put these rates behind your own key with spend guardrails, see the model catalog for per-model prices and the docs for the OpenAI-compatible endpoints.
When caching won't save you#
- Short prompts. Under 1,024 tokens,
cached_tokensis zero by definition. Chatbots with tiny system prompts get nothing. - Unstable prefixes. Anything dynamic early in the prompt — timestamps, user names, request IDs, reordered tool schemas — silently kills the hit rate. The cache never errors; it just misses.
- One-shot workloads. Caching discounts repeats. A single long document processed once gets zero benefit; that's a Batch API workload (50% off, 24-hour turnaround), not a caching workload.
- Strict data-retention requirements. The
24hretention tier stores tensors outside GPU-local memory. If your org needs zero data retention, stay onin_memoryand accept the 5–10 minute window.
OpenAI prompt caching FAQ#
Do I need to change my code to enable prompt caching?
No — on GPT-4o through GPT-5.x it is automatic for prompts of 1,024 tokens or more. No markers, no parameters, no SDK upgrade. Verify with usage.prompt_tokens_details.cached_tokens. (On GPT-5.6+, explicit breakpoints are available but optional.)
How much does cached input actually cost per model? 50% off on GPT-4o and the o-series, 75% off on GPT-4.1 and o3/o4-mini, 90% off on the GPT-5 family, 95% off on GPT-6.1 Sol. In dollars: GPT-5.5 cached input is $0.50/M, GPT-4.1 is $0.50/M, GPT-4o is $1.25/M. Prices checked 16 October 2026.
Why is my cached_tokens always zero?
Almost always one of four causes: the prompt is under 1,024 tokens; something dynamic (timestamp, user ID) sits in the early prefix; tool definitions serialize differently between calls; or requests are landing on different machines (try a stable prompt_cache_key). All four fail silently.
Does OpenAI charge to write the cache like Anthropic does? Not on GPT-4o through GPT-5.x — writes are free. On GPT-5.6 and later, cache writes cost 1.25× the input rate, so the Anthropic-style break-even math starts applying there.
Does prompt caching change what the model outputs? No. The cache stores intermediate computation for the prompt prefix, not generated tokens — sampling proceeds normally. Latency typically drops because the prefix isn't recomputed (OpenAI cites up to ~80% time-to-first-token reduction at long contexts).
Discount tiers, mechanics, and list prices checked 16 October 2026. Providers change defaults and prices; OpenAI's pricing page is the authority for list prices, and the model catalog for prices on Tokens.
Sources: OpenAI prompt caching docs · OpenAI API pricing · LiteLLM prompt caching docs · OpenAI Prompt Caching 201 cookbook notes · OpenAI API pricing breakdown 2026 · GPT-5.5 pricing guide · GPT-4 family pricing breakdown · USD/BDT rate · tokens.bd models · tokens.bd docs
