Prompt Caching: The 90% Discount Most LLM Apps Aren't Using
The biggest underused lever on an LLM bill, explained across all three major providers — including where it backfires.
This is part of our guide to optimizing LLM API costs.
Most production LLM applications send the same block of text at the start of every single request. A system prompt with instructions and rules. A set of few-shot examples. A retrieved document or a fixed knowledge base. That prefix is often the largest part of the prompt — and it is identical from one call to the next.
Prompt caching is the mechanism every major provider now offers to stop charging full price for that repetition. When it’s configured correctly, the repeated portion of a prompt is billed at roughly one-tenth of the normal input rate. For an application whose prompts are mostly static context, that translates into an input-cost reduction approaching 90 percent, with no change to the model, the output, or the product behavior.
It is the single largest cost lever available to most teams, and it is frequently left switched off — either because caching is opt-in and nobody enabled it, or because the prompt is structured in a way that prevents the cache from ever being hit.
What caching actually does
An LLM processes a prompt as an ordered sequence of tokens. Providers cache from the front: they store the computed state of a prefix and reuse it on a later request that begins with the exact same tokens. The moment the sequences diverge, caching stops for the rest of the prompt.
Two consequences follow directly from that design, and both matter more than the headline discount:
Order is everything. Static content has to sit at the beginning of the prompt for caching to work. If a request starts with a unique element — a timestamp, a user ID, a session token — the cache is busted at token one and nothing downstream is reused. The fix is structural: put everything stable (system prompt, instructions, fixed examples, retrieved context) first, and everything variable (the user’s actual message) last.
Prefixes must match exactly. A single changed character early in the prompt invalidates the cache from that point forward. Injecting the current date into a system prompt, or reordering a set of examples between calls, silently defeats caching even when 99 percent of the text is identical.
What it costs across the three major providers
The economics differ by provider, and the differences change which workloads benefit. The figures below are per million tokens and current as of August 2026; provider pricing changes often, so confirm against the live pricing page before committing a budget.
| Provider | Cache read (input) | Cache write | Notes |
|---|---|---|---|
| OpenAI (GPT-5.4–5.6 families) | ~10% of standard input | No separate write fee | Automatic — no code change; typically applies to prompts over ~1,024 tokens |
| Anthropic (Claude) | 10% of standard input | 1.25× input (5-min) or 2× input (1-hour) | Explicit — you mark what to cache with cache-control breakpoints |
| Google (Gemini) | 10% of base input | Storage billed per hour | Implicit caching automatic; explicit context caching adds an hourly storage fee |
Three practical distinctions come out of that table.
OpenAI’s caching is automatic and free to enable — repeated prefixes are discounted without any code change, which means the main risk is simply structuring prompts so the cache is never hit. Anthropic’s caching is explicit and has a write premium: the first request that populates the cache costs more than a normal request (1.25× or 2× input, depending on how long you want the cache to live), and every subsequent read costs 10 percent. Google’s explicit caching adds a storage cost billed by the hour, which changes the math for content you cache but read infrequently.
The math that decides whether it helps
Whether caching saves money is a break-even question, not a guarantee — and the write premium is where teams get it wrong.
Because populating a cache can cost more than a normal request, caching a prefix that gets read only once or twice is a net loss on providers that charge for writes. Caching pays off when the same prefix is read many times before it expires. The rule of thumb: the more requests that reuse the same prefix within the cache’s lifetime, the larger the saving; a prefix read once is worse than not caching at all.
A concrete illustration. Consider an application with a 5,000-token static system prompt, serving 50,000 requests a day, on a model priced at $3 per million input tokens. The system prompt alone accounts for 250 million input tokens a day, or $750. At a 90 percent cache-read discount, the repeated portion drops toward $75 — before counting the smaller, variable user input on top. The saving scales directly with how static and how large the prefix is.
The workloads that benefit most share a profile: a large, fixed prefix (long system prompt, few-shot examples, or retrieved context) reused across high request volume in a short time window, so reads land before the cache expires.
Where it backfires
Three failure modes turn caching from a saving into waste or a non-event:
Unique-per-request prompts. If every request is genuinely different from the first token — no shared system prompt, no fixed context — there is no prefix to cache and the feature does nothing. Interpolating variable data near the top of the prompt produces the same result by accident.
Write-heavy, read-light patterns. On providers that charge a write premium, caching content that is rarely reused costs more than skipping the cache. Nightly batch jobs with distinct prompts each run are a common example.
Expiry gaps. Caches live for minutes by default. An application with sparse, irregular traffic may see each request arrive after its predecessor’s cache has already expired, paying the write premium repeatedly and never collecting the read discount.
Turning it into a reliable saving
Caching only reduces a bill if the cache is actually being hit, and the hit rate is invisible from a provider dashboard that reports a single monthly total. The practical way to verify it is to track cost per request over time and watch the number fall after caching is introduced — and to watch it stay down, since a later prompt change can silently break the cache and quietly restore the full rate.
That kind of per-request, real-time visibility is what LLMtrack is built for: it records the computed cost of every call as it happens, so a caching change shows up as a measurable drop within seconds rather than a guess confirmed on next month’s invoice. Start tracking free and see your cost per request before and after you turn caching on.