Cost Optimization

How to Calculate Your Real LLM API Cost (The Numbers Most Guides Get Wrong)

The naive estimate prices the user's message and the visible answer. The real request also carries a system prompt, history, reasoning tokens, cache writes and retries. A worked example where the real cost is 3.9× the estimate, the full formula, and a calculator.

6 minBy LLMtrackUpdated Rates checked 17 Nov 2026
A chat request on GPT-6.1 Terra at 100,000 requests a month: the naive estimate is $190, the real cost with system prompt, history, reasoning tokens and retries is $733, 3.9 times higher.
Short answer: real cost per request = (system prompt and tools, at the cached or uncached rate) + (history and retrieved context) + (user message), all at the input rate, + (visible answer + reasoning tokens) at the output rate, × (1 + retry rate). Then check for long-context tiers, service tiers and regional pricing. In the worked example below, a request estimated at $0.0019 really costs $0.0073 on GPT-6.1 Terra: 3.9× the estimate, mostly from reasoning tokens and history.

The usual way to estimate an LLM feature is to take the length of a typical user message and a typical answer, multiply by the rates on the pricing page, and multiply by the number of requests. It gives a clean number, and it's usually low by a factor of two to four.

The request that actually goes out carries much more than the message and the answer, and some of what you're billed for never appears in the answer at all.

This post is part of the "Understand what drives your bill" section of the complete guide to LLM cost optimization.

The naive estimate

Take a chat feature on GPT-6.1 Terra ($1 per million input tokens, $5 output, at OpenAI's published schedule on 17 Nov 2026). A typical user message is 150 tokens and a typical answer 350:

150 × $1/1M + 350 × $5/1M = $0.0019 per request, or $190 a month at 100,000 requests.

The real request

Here's the same request as it actually goes out:

PartTokensRateCostShare
System prompt and tools2,00080% cached$0.000588%
History and retrieved context1,500$1$0.001520%
User message150$1$0.000152%
Visible answer350$5$0.0017524%
Reasoning600$5$0.003041%
Retries, 5%$0.000355%
Total$0.00733100%

That's $733 a month against the $190 estimate. The part the naive estimate counted (message and visible answer) is 26% of the real cost.

The numbers are a worked example, not a measurement. Your mix will differ, which is why the calculator below lets you enter your own.

Real cost per request calculator

Fill in one request as it really goes out, and compare it with the naive estimate (the user's message and the visible answer at list price). Rates in USD per million tokens, checked 17 Nov 2026.

Cache misses are billed as cache writes where the provider charges for them. Each retry is counted as one full extra request.

Each hidden part, and where to find it

System prompt and tools. Billed as input on every request. Tool definitions count too. Caching discounts it heavily when the prefix is identical across requests, but on OpenAI's GPT-5.6-and-later models and on Anthropic, the requests that miss the cache pay 1.25× input to write it. What your system prompt costs and what tool definitions cost go deeper.

History and retrieved context. Chat APIs resend the whole conversation every turn, and RAG features paste retrieved chunks into every prompt. Both grow with usage in ways a single-request estimate misses. See conversation history costs and what a RAG query costs.

Reasoning tokens. Reasoning models think before they answer, and that thinking is billed at the output rate. OpenAI and Anthropic include it in the output token count; Gemini reports it separately as thinking tokens. It's often the largest line, and it changes with the effort setting. How to reduce output tokens covers the settings.

Retries. A retried request is sent and billed again in full. Timeouts, rate limits and malformed output (especially truncated JSON) all trigger retries, and a retry loop without a cap can multiply a feature's cost. How retry logic inflates the bill has the patterns.

Pricing rules that change the rate itself

Even with the right token counts, the rate isn't always the one in the headline:

  • Long-context tiers. Some models charge more once a prompt passes a threshold. GPT-6 Luna goes from $0.10 and $0.50 to $0.20 and $0.75 above 272,000 tokens; Gemini 3.1 Pro from $2 and $12 to $4 and $18 above 200,000; Claude Haiku 5.5 from $0.10 and $0.50 to $0.50 and $2.50 on the whole request above 100,000.
  • Service tiers. Batch and Flex cost half the standard rate on OpenAI and Gemini, and Anthropic's Batch API is also half price. OpenAI's Fast tier and Anthropic's fast mode cost twice the standard rate.
  • Regional endpoints. On Amazon Bedrock, regional endpoints for Claude and OpenAI models cost 10% more than global ones.
  • Scheduled price changes. Gemini 3.6, 3.7 and 3.8 Flash double on 1 January 2027. A December estimate is wrong for January. The Gemini Flash increase has the numbers.

Measure it instead

An estimate is a starting point. The real cost per request comes from the usage data each provider returns with every response: input, cached and output tokens, and on reasoning models the reasoning count.

LLMtrack records that usage after each call, prices every part at the model's rate (cached, cache-write, reasoning and long-context tiers included), and tags it with the feature and customer that made it. That gives you the real cost per request per feature, from production traffic rather than a spreadsheet. You can start on the free plan, or use the LLM cost calculator to estimate before you ship.

import { LLMtrack, fromOpenAIUsage } from 'llmtrack';

const llmtrack = new LLMtrack({ apiKey: process.env.LLMTRACK_API_KEY });

const started = Date.now();
const res = await openai.chat.completions.create({ model: 'gpt-6.1-terra', messages });

llmtrack.track({
  provider: 'openai',
  model: res.model,
  feature: 'chat-reply',
  customerId: account.id,
  latencyMs: Date.now() - started,
  ...fromOpenAIUsage(res.usage), // cached and reasoning tokens priced at their own rates
});

FAQ

Add up every input token (system prompt, tools, history, retrieved context and the user message) at the input rate, with cached tokens at the cached rate, then add the visible answer and any reasoning tokens at the output rate. Multiply by your retry rate on top.

Their count does, in the usage data, even though their content doesn't appear in the answer. OpenAI reports them as reasoning_tokens inside the output details, Anthropic includes them in output_tokens, and Gemini reports them as thoughtsTokenCount.

Add the embedding of the query (usually negligible), the retrieved chunks as input tokens, the system prompt, and the answer, plus any extra calls like query rewriting. The retrieved chunks are usually the biggest input line.

Yes. Every retry is billed as a full new request. Measure your actual retry rate per feature and add it to the estimate, and cap retries so a failing dependency can't multiply the bill.

Usually because the estimate only counted the user's message and the visible answer. System prompts, conversation history, tool definitions and reasoning tokens are all billed, and often make up most of the cost.

Sources

Checked on 17 Nov 2026:

See your real cost per request

Free plan, every feature, no card.

Start tracking free →