Reasoning Tokens: The Invisible Line Item in Every New AI Model

For most of the last two years, choosing an AI model was a single decision. You picked GPT or Claude or Gemini, wired it in, and the per-token rate on the pricing page was roughly what you paid. The newest generation of models quietly broke that assumption. The question is no longer just which model — it's which model, at what effort. And the answer to the second half of that question is now the single biggest swing factor in what an AI feature costs to run.

The reason is a token type that didn't meaningfully exist in production budgets eighteen months ago: the reasoning token. Every current flagship — GPT-5.6, Claude Opus 5 and Sonnet 5, Gemini 3, Grok 4.5 — thinks before it answers, generating an internal chain of reasoning that never appears in the final response but is billed at the full output rate. On a hard request, a model can spend tens of thousands of tokens thinking and a few hundred answering. You pay for all of it, and on most providers you can't see the part you paid the most for.

This is part of the broader problem of what actually drives your LLM bill. Reasoning tokens are the newest and least-understood entry on that list.

What a reasoning token actually is

A reasoning model does its thinking in tokens. Before it writes the answer you asked for, it generates a private chain-of-thought — working through the problem, checking itself, exploring alternatives — and only then produces the visible reply. Those intermediate tokens are called reasoning tokens (OpenAI's term) or thinking tokens (the more common name), and the crucial billing fact is simple: they are charged at the output rate.

There is no separate, cheaper meter for reasoning. A reasoning token costs exactly what an output token costs, which on a flagship model is the most expensive token you can buy — several times the price of an input token. A model priced at $30 per million output tokens charges $30 per million reasoning tokens too.

The providers handle visibility differently, and the difference matters. OpenAI's reasoning tokens are hidden: the API returns the final answer plus a short summary, and the full trace stays internal — but every token in it is still on the bill. Anthropic's extended thinking is returned to the developer as visible thinking blocks. Either way, the tokens count. The usage object on a reasoning-model response breaks them out explicitly — a field like reasoning_tokens sitting inside the completion details — which means a request that returns a 300-token answer can quietly report several thousand tokens consumed.

Every new model now ships with a dial

The defining shift of 2026 is that reasoning depth became a setting. In 2025, more intelligence meant switching to a bigger model. In 2026, it increasingly means turning a dial on the same one — and that dial moves your cost as much as a model change used to.

The controls vary by provider, and the details are worth knowing because they determine your floor:

• OpenAI (GPT-5.6 Sol, Terra, Luna). A reasoning_effort parameter, extended in the latest generation up to a max setting, plus a heavier "ultra" mode that farms a task out to parallel sub-agents. A notable quirk: GPT-5.6 treats effort as a ceiling rather than a floor — on a prompt it judges easy, it may perform no reasoning at all, even when effort is set high. That's good for your bill, but it means effort settings don't guarantee spend in either direction.
• Anthropic (Claude Opus 5, Sonnet 5). A full effort ladder — low, medium, high, xhigh, max. The catch worth flagging: on the Claude 5 models, thinking is always on and cannot be disabled. Passing "none" does not turn reasoning off; the model reasons at its server-side default, which skews high. The older manual token-budget control was retired on recent Opus versions in favor of this adaptive approach.
• Google (Gemini 3 Flash and Pro). Discrete thinking levels — Flash exposes minimal, low, medium, and high; Pro drops the minimal option — alongside a separate Deep Think mode for the hardest problems.
• xAI (Grok 4.5). Reasoning is on by default, with a configurable effort parameter in the API.

The practical consequence is that "which model should I use?" has quietly become "which model, at what effort?" — and the labs themselves now frame the comparison in terms of cost per completed task rather than raw capability. At maximum reasoning, the top models land within a point or two of each other on independent intelligence benchmarks; the spread that remains is largely a spread in how many tokens each one burns to get there.

Even the newer challengers illustrate the point. One recently launched model shipped with only a single "max" effort level, and a trivial test prompt — generate an SVG of a pelican on a bicycle — consumed over 16,000 output tokens for a 25-cent result, purely because it had no way to think less. The absence of a dial is itself a cost decision.

Why this is a bill problem, not just a feature

Reasoning is genuinely useful. On hard, multi-step, agentic work, more thinking reliably produces better results — some evaluations show agent task performance climbing from the low fifties to the low eighties as effort scales up. The problem isn't that reasoning costs money. The problem is that the cost is variable, invisible, and defaults high.

Consider the shape of a typical cost. A request that generates 10,000 thinking tokens before a 200-token answer, on a model at $30 per million output tokens, spends about $0.30 on reasoning and roughly $0.006 on the visible reply. The thinking cost fifty times more than the answer — and none of it appeared in the response your code received. Multiply that by a pipeline running tens of thousands of requests a day and the reasoning overhead becomes the dominant term in the bill, while every dashboard your team looks at shows only a normal-looking response.

This is exactly the kind of cost that stays hidden until someone measures it, in the same family as the system prompt you re-send on every request and the runaway agent loop that spends a month of budget in an hour. It doesn't show up as an error, a latency spike, or a traffic anomaly. It shows up on the invoice.

More thinking isn't always better

Here's the part that turns reasoning tokens from a cost of doing business into an optimization opportunity: on a large share of real workloads, higher effort buys little or nothing.

Reasoning pays off on genuinely hard problems — novel logic, long agent chains, complex code. But on knowledge retrieval, classification, extraction, formatting, and routing, several evaluations find that reasoning effort barely moves the outcome. In one study across three frontier models, accuracy at low, medium, and high effort clustered within a razor-thin band, and the ranking wasn't even consistent — one model scored best at low effort. When the answer doesn't improve, every reasoning token is pure waste billed at the output rate.

That is the actionable core of the whole topic. The default on the newest models leans toward more thinking, because labs are optimizing for benchmark scores, not your budget. Matching effort to task difficulty — high effort only where it demonstrably helps, minimal or off where it doesn't — is one of the largest cost levers available on current models, and it lives entirely in a parameter you were probably leaving at its default. It sits right alongside the other reductions that don't require touching your prompts.

You can't tune a dial you can't see

The obstacle is visibility. A provider's usage dashboard reports one aggregate figure, updated once a day, with no breakdown of how much of it was reasoning, which feature generated it, or whether a given effort setting is earning its cost. Industry surveys put the share of organizations that track AI spend at the level of individual transactions at barely one in five — which means most teams are paying for reasoning they can neither see nor attribute.

The information exists; it's just not surfaced. Every reasoning-model response reports its reasoning-token count in the usage object. Capturing that number on each request — alongside the model, the feature, and the effort level — is what turns an invisible line item into something you can actually manage: you can see which features are reasoning-heavy, watch what happens to cost when you lower effort on a task that doesn't benefit, and catch a default that quietly doubled your bill after a model upgrade.

That per-request visibility is what LLMtrack is built to give you. It records the computed cost of every call the moment it completes — output tokens, reasoning included — and groups spend by feature and model, so the cost of thinking stops being a mystery you reconcile at month-end and becomes a number you can see and act on in real time. If reasoning models are anywhere in your stack, tracking your usage is the difference between tuning the dial deliberately and paying for whatever the default happens to be.

Model behavior, effort controls, and pricing referenced here are current as of August 2026 and change frequently. Confirm current rates and reasoning settings on each provider's official documentation before making a budgeting decision.