[DigitalToday reporter Seulgi Son] "Using a combination of multiple models has already become routine. Users just do not realise it."

Lee Jun-beom (이준범), a researcher at Ravlub, said in an interview that generative AI services already assign roles to models of different sizes. Small models are used for relatively simple tasks such as search, information gathering and summarisation, while frontier models are left to make key judgments and generate outputs.

A representative example is a coding agent. Lee said the smallest model gathers information by reading files or searching the internet, while a larger model handles further exploration based on that information or writes important code.

As agents spread, the need for such division of labour is growing. As AI repeatedly calls models in the process of finding, reading, judging and acting on information, inference costs can rise quickly if the highest-performance model is used for every task.

Lee is a developer who has studied Korean natural language processing (NLP) and open language models. He developed Korean-based open models such as KoAlpaca and Gemma-KO, and has recently expanded his research into small models, agents and inference efficiency.

Small models and open models are distinct concepts. Small models are classified by size and compute, while open models refer to models whose weights and other elements are made public so they can be used externally. Among open models, there are both small and large models.

Lee said the direction of development for small models has also changed. "Recently demand has shifted toward training focused on intelligence rather than knowledge," he said. "Small models are evolving to search the web when needed and to read given documents and answer from within them."

Still, small models are less capable than large models at adapting to newly introduced tools or tasks without additional training. Lee said he studied 'MCP Sidecar sLM' last year to address this. When small open models could not use the Model Context Protocol (MCP) proficiently, he attached a separate small model trained in how to use MCP to support tool use.

"The goal was to add an even smaller model that understands MCP itself and plays a sidekick role," he said.

As open models released since then have come with built-in MCP or tool-using capabilities, the need for that research has diminished. He said the approach can still be used, with a small auxiliary model added to handle a new tool or interface and respond quickly, rather than retraining an entire existing model.

Lee rejected forecasts that small models will replace frontier models. "Since 2018, the rule that has not been broken is scaling," he said. "That models become smarter as they get bigger has not been disproved yet."

Models of the same size can deliver higher performance than a few years ago thanks to advances in data and training methods, he said. But when comparing the latest models at the same point in time, larger models still have higher intelligence, he said.

Upsizing is also a prominent trend recently among open-model developers. In the past, models with 100 billion to 200 billion parameters were classified as large, but models with trillions of parameters have recently emerged. Lee said that alongside the upsizing of frontier models, the practice of assigning roles across models of different sizes will spread further.

Such changes are also reflected in corporate demand for AI infrastructure. "In the past, demand centred on pre-training or training, but recently there has been a lot of demand for inference and reinforcement learning," he said. He said more companies are weighing not just building GPU infrastructure itself but also which models can be used on it, by how many users and at what speed.

In corporate settings, the standard for what counts as a 'small model' also changes. On personal PCs or smartphones, models with several billion parameters are seen as small, but in companies, open models with tens to hundreds of billions of parameters can be treated as relatively small models. Lee cited the size of models that can run on a single NVIDIA personal AI supercomputer, the 'DGX Spark', as a reference point for operating models in a local environment.

In training, fast communication across multiple GPUs and servers was important, but in inference, memory and key-value (KV) cache management to handle many users' requests at once are becoming more important. Lee said he is also researching diffusion-based language models that generate multiple tokens at once to increase inference speed.