In a new research paper, a team from NVIDIA Research and the Georgia Institute of Technology presents a compelling argument that small language models (SLMs) are the future of AI agents, challenging the industry’s focus on ever-larger models. The paper, “Small Language Models are the Future of Agentic AI,” posits that for the majority of tasks performed by AI agents, SLMs are not only sufficiently powerful but also more suitable and economically viable.
The research suggests a paradigm shift from a monolithic model architecture to a heterogeneous system where multiple models, both large and small, are used for different purposes. This approach reserves large language models (LLMs) for complex, open-ended tasks while leveraging SLMs for the repetitive, specialized, and domain-specific functions that make up the bulk of agentic workloads. According to the paper, SLMs can achieve performance on par with LLMs on tasks like tool calling and instruction following, while reducing computational volume by 10-30 times. This efficiency allows for significant reductions in latency, memory, and operational costs.
The economic and operational benefits of this shift are substantial. The paper highlights that using giant LLMs for routine tasks is economically inefficient and environmentally unsustainable at scale, akin to “renting a rocket just to deliver a pizza.” The lower costs and computational demands of SLMs could democratize AI agent development, allowing for on-device deployment on consumer hardware and enabling a wider range of developers and businesses to build and deploy solutions. This shift could also address current market inefficiencies, where a large amount of capital has been invested in centralized, large-model infrastructure.
