The AI Cost Paradox: Tokens Are 97% Cheaper, So Why Are Bills Exploding?

In early 2026, the IT director at a machinery manufacturer in southern Germany stared at a ChatGPT Enterprise invoice that dwarfed the previous year's bill. Here's the strange part: between GPT-4 and GPT-5.6, the price per million tokens had dropped roughly 97 percent, according to OpenAI's own pricing pages. His CFO had read that exact headline — and expected the cost center to shrink accordingly. Instead, the IT director had to stand up in the quarterly review and explain why spending had gone up despite the price collapse. He couldn't.

This scene is playing out in finance departments across the country right now. Trade press keeps reporting collapsing token prices, the internal cost center shows the opposite, and nobody in the building can cleanly explain where the gap comes from. Even more uncomfortable: there's no basis for saying whether that extra spend is actually buying something valuable or just growing unchecked. We keep running into this exact gap between the headline and the invoice in client projects — usually right after the bill has already landed on someone's desk in a quarterly meeting.

This article breaks down the cost mechanics behind that paradox — starting with the basic math, moving through agent-driven usage patterns, and ending with the cost buckets that never show up on any API bill. And it lays out a concrete approach finance and IT teams can use to make usage, spend, and output quality visible at the workflow level, instead of staring at a metric that, by 2026, barely tells you anything useful.

At first glance, the numbers are impressive. According to OpenAI's pricing sheet, GPT-4 cost roughly $30 per million input tokens and $60 per million output tokens when it launched in 2023. By early 2026, GPT-5.6 runs at a fraction of that — a price drop of roughly 97 percent per million tokens across model generations. No other line item in enterprise IT has gotten this much cheaper this fast. Not storage, not compute, not SaaS licensing.

But anyone concluding that the AI bill should drop by 97 percent is making a basic math error. The total bill still follows the simplest formula in finance:

Total Cost = Unit Price × Volume

Unit price dropped 97 percent. For the bill to stay flat, token volume could grow by a factor of up to 33. That's exactly where things go sideways: in many companies, usage grew faster than that. Not by a factor of 5 or 10 — but by 50, 100, or more. The result is a bill that keeps climbing despite a historic price collapse.

Here's the key point for the boardroom conversation: the rising bill is not a pricing problem. OpenAI didn't raise rates. There are no hidden fee adjustments, no contract traps. The bill grows purely through volume. Economists have a name for this pattern — the Jevons paradox: when a resource gets cheaper, consumption rises disproportionately because use cases that were previously unthinkable suddenly become economically viable. It happened with coal in the 19th century. It's happening with tokens in 2026.

So the real question isn't "Why did AI get more expensive?" It's: Why are we suddenly burning through a hundred times more tokens? The answer lies in a fundamental shift in usage patterns — away from simple chat queries and toward something entirely different.

In 2023, typical AI usage looked simple: an employee types a question, the model answers, done. One prompt, one API call, a few thousand tokens. By 2026, that same interaction looks structurally different — and this shift is exactly what's driving the cost paradox.

Modern agentic workflows break a single user request into a chain of model calls. An agent tasked with something like "check this supplier quote against our master agreements" typically works through the following steps:

1. Planning: The model breaks the task into sub-steps — first call.
2. Tool calls: For each sub-step, the agent queries databases, documents, or APIs and processes the results — often ten to thirty additional calls.
3. Verification: Intermediate results get checked against the original task — more calls.
4. Synthesis: Only at the end does the single answer the user actually sees get generated.

A single prompt can easily turn into 30 to 50 API calls. The user never sees any of this — the invoice does. Just how much this shift is already reshaping engineering teams becomes clear in our look at agentic coding in the daily life of a CTO, where individual developer sessions can burn through millions of tokens.

The second driver is the context window. Current models like GPT-5.6 or Claude Sonnet 5 can process over 1 million tokens of context per request. That sounds like a pure performance feature — but it's actually a cost feature. Feed the model your entire contract folder, your full codebase, or your complete CRM segment, and you pay for every single one of those tokens. On every single call. In an agent loop with 40 calls, that same context can get processed — and billed — 40 times over in the worst case.

The third factor is reasoning models. Chain-of-thought processes and self-correction steps generate internal "thinking tokens" that users never see but that get billed in full. A response with 500 visible tokens can rack up 20,000 reasoning tokens behind the scenes. The model is thinking out loud — and the customer pays for that inner monologue.

Taken together, these three factors drive the volume explosion described earlier: more calls per task, more context per call, more internal tokens per response. And as if that weren't enough, these repeated calls generate additional cost blocks that never show up on any token invoice.

If you're only looking at your OpenAI invoice, you're seeing a fraction of your actual AI costs. Three cost blocks stay invisible on a standard API bill — and in practice, they often add up to several times the raw token cost.

Agents fail. A tool call times out, an API throws an error, a model returns an unusable format. What happens next? The orchestration layer tries again — and every retry burns tokens. Error-handling loops that reprocess the same context three, four, or five times never show up on an invoice as a "failed attempt." They look like normal usage. In poorly built pipelines, retries can account for 20 to 40 percent of total consumption without anyone noticing — simply because nobody tracks what share of calls succeeded on the first try. In practice, this is one of the first things that surfaces when auditing a client workflow: a clear look at just how big the gap is between perceived usage and actual usage.

The second invisible cost block is organizational: teams running their own AI subscriptions and API keys outside of centralized contracts. Marketing puts a tool on a corporate card, sales trials an agent through an individual plan, and engineering spins up private accounts. According to a Cyberhaven analysis, more than 70 percent of AI usage at many companies runs through private or unapproved accounts — invisible to both IT and Finance. These expenses get scattered across dozens of cost centers under line items like "Software" or "Miscellaneous," and never show up in any AI budget overview. We took a deep dive into just how far this problem now reaches in our analysis of shadow AI in business units.

The third cost block: no production AI workflow runs on model calls alone. Add vector databases for document search, embedding costs to prep your own data, orchestration platforms, monitoring tools, and logging storage. Each of these components gets billed separately — by different vendors, on different invoices, often landing in different cost centers. A workflow with $500 a month in token costs can easily generate another $1,500 in supporting infrastructure that never gets labeled "AI" anywhere.

This is usually where leadership pushes back: "Fine — but if tokens are 97% cheaper, shouldn't each individual task at least cost less than it used to?" Sounds logical. It isn't.

Here's the uncomfortable core of the issue — and a thesis that meets resistance in plenty of boardrooms: Token price is nearly irrelevant for 2026 budget planning. Until a company measures what a completed task actually costs, price per million tokens is a metric with zero control value. You can use it to comment on headlines, but you can't run a budget with it.

The root cause is confusing two different control metrics:

Here's the mechanics behind it: a task that got done in 2023 with a single GPT-4 call and 5,000 tokens cost about 15 cents. The same task in 2026 runs through an agent with 40 calls, reasoning steps, and a massive context window — processing 2 million tokens. Even at GPT-5.6's drastically lower price, that task can end up costing more than it used to. The more complex, multi-step task running on the cheap model beats out the simple task running on the expensive model — on the cost side, that is.

That doesn't automatically mean the agent is the worse choice. It might deliver a noticeably better result, check more sources, and make fewer mistakes. But answering the real question — is the extra spend worth it? — requires looking at cost and outcome together, per task. Our calculation on AI-generated product descriptions across 1,500 SKUs shows how quickly even modest use cases can develop unexpected cost dynamics.

The boardroom expectation of "price down, bill down" is therefore structurally wrong — not because prices didn't fall, but because what gets consumed per task has changed fundamentally. Any team that doesn't actively correct this expectation in its reporting will keep facing the same awkward explanations every single quarter. What that correction looks like in practice becomes clear once you dig into an actual cost review.

Consider an illustrative example — the numbers are deliberately built as a model calculation, but the pattern behind it is one we see regularly in client projects: A mid-sized industrial supplier with roughly 400 employees automates its quote review process. Incoming supplier quotes get checked by AI against master agreements, price lists, and compliance requirements before procurement signs off on them.

The starting point in Finance: after switching to the new model generation, the price per token dropped dramatically — yet the monthly AI bill for the overall process still climbed from roughly $850 to roughly $3,100. To leadership, that looked like a project spiraling out of control.

The decisive move was that Finance abandoned the aggregate bill and instead broke costs down per workflow. That's when the picture changed:

• The number of quotes reviewed had jumped from 300 to 850 per month — because procurement increasingly trusted the system and started routing even smaller quotes through it.
• Token consumption per quote had risen because the new agent no longer just read the quote text — it also pulled master agreements and pricing history into context and cross-checked every flagged item in a verification step.
• At the same time, manual rework had collapsed: before the switch, procurement had to manually correct roughly one in three AI results; afterward, that dropped to roughly one in twelve.

The company's controller distilled it into a line that's now quoted in every cost review since:

"We spent months arguing over the wrong number. The bill had gone up by roughly 3.6x — but the cost per cleanly reviewed quote had actually gone down. We only saw that once we stopped staring at the total."

So the aha moment wasn't a cost reduction — it was a reframe. Most of the increased usage was justified, because it translated into measurably better outcomes and less expensive staff time spent on rework. Part of the increase, though — redundant context transfers inside the agent loop — wasn't justified, and the team went on to optimize that specifically.

The real lesson here: this insight didn't happen by accident. It came from a specific way of measuring things. And that kind of measurement can be built systematically — instead of stumbling into it once, anecdotally, and hoping it happens again.

What cloud teams learned from FinOps a decade ago is now catching up with AI budgets: without granular mapping of usage back to the teams driving it, every cost discussion is just guesswork. The good news is that the technical lift is manageable. Four steps are enough to get started.

1. Make tagging mandatory. Every API call gets metadata: team, use case, workflow, environment (test/production). All major providers support this through headers or project structures. Without tagging, any later analysis is impossible — which is why this rule belongs in your development guidelines before the first production workflow goes live.
2. Bring cost and quality into one dashboard. A dashboard that only shows token consumption creates panic. A dashboard that only shows output quality creates blind spots. Only the combination — usage, cost, and a quality metric like rework rate or success rate per process — makes real evaluation possible.
3. Set budget alerts per workflow, not globally. A single global spending cap is worthless: it only fires after the damage is already done, and it tells you nothing about which process is driving the spend. Thresholds per workflow — say, "quote review: alert above 120 percent of the monthly average" — let you pinpoint outliers immediately.
4. Establish a monthly review routine. Visibility without a rhythm fizzles out fast. A fixed monthly meeting where finance and IT walk through the top five workflows by cost takes 30 minutes and replaces hours of ad-hoc scrambling after the next surprise invoice.

For most companies, setting up this baseline takes only a few person-days — provided the workflows run through a centrally controlled integration layer. Where that layer doesn't exist yet, building it is the real first step; we break down what that architecture can look like in our Software & API Development practice.

Visibility alone won't shave a single dollar off your bill, though. It only lays the groundwork. Turning transparency into actual cost control requires rules — both organizational and technical.

In 2026, the biggest savings won't come from renegotiating vendor contracts. They'll come from answering a different question: which model handles which task — and who's accountable for the cost. Three governance rules have proven themselves in practice.

The most expensive habit in AI projects is the reflex to send every request to the most powerful model available. But most tasks in typical workflows are trivial: classifying, extracting, formatting, summarizing. Small, low-cost models — Flash or Lite variants, for example — handle these just fine, at a fraction of the per-token price. A routing layer that sorts requests by complexity and only sends genuine reasoning tasks to models like GPT-5.6 or Claude Sonnet 5 cuts costs on many workflows by 50 to 80 percent — with no measurable quality loss on the simple tasks. The prerequisite is the visibility described in the previous section: only teams that know which task types make up a given workflow can route requests intelligently.

The second lever targets the multiplier problem in agent loops head-on: when the same context block—system prompt, contract library, product catalog—gets retransmitted with every call in a loop, you're paying for it fresh each time. Prompt caching, now supported by all major providers, stores frequently used context blocks and bills them at a significant discount on reuse. It's also worth asking the architectural question of whether the agent truly needs the full context at every step—or whether targeted reloading of specific sections would do the job. In practice, combining both measures cuts per-loop context costs substantially.

The most effective organizational rule is also the most unpopular one: business units cover their own AI costs. As long as AI spending disappears into a central IT budget, no department has any incentive to use it sparingly — that's the same tragedy-of-the-commons logic that fuels the shadow AI problem in the first place. A chargeback model that allocates tagged costs back to department budgets based on actual usage changes behavior within a few months. Add quarterly cost reviews where each department has to justify its most expensive workflows, and you get a real feedback loop: visibility creates accountability, accountability creates pressure to optimize, and optimization becomes measurable again through the dashboards. We break down how these governance building blocks fit into a broader automation strategy in our AI & Automation practice area.

The sequence here isn't optional: visibility first, rules second. Governance without measurement just produces bans that get routed around through shadow usage. Measurement without governance produces nice-looking dashboards with zero impact. Put both together, and an uncontrolled cost center turns into an investment you can actually manage.

That resolves the paradox from the headline: by 2026, the per-token price has become a footnote. The real cost drivers are agent loops, expanding context windows, reasoning overhead, retry loops, and untagged usage — factors no price drop on earth can offset as long as nobody's measuring them. Classic price comparisons between model generations are answering the wrong question. The metric that actually matters is cost per completed task, broken out by workflow — paired with a quality dimension, without which no increase in usage can be properly judged.

For finance and IT teams, that translates into a concrete next step: over the next two weeks, pick a single core AI workflow — whichever one has the highest bill or the biggest strategic weight — and track two numbers for it separately: token consumption per completed task, and a quality metric like the rework rate. That's all the first building block of a cost-per-outcome dashboard needs. Based on multiple client engagements, we know that even this one workflow generates enough insight to walk into the next budget conversation with leadership on entirely different footing — with numbers instead of headlines.

Companies that take this step now gain a structural edge over the competition: while others are still puzzling over rising bills, these teams already know which extra euro of usage pays off — and which one doesn't.