More booth at SC 2026. [Photo: More]

AI infrastructure solutions company More said on Thursday it unveiled TIDE, a framework that improves inference efficiency for large language models (LLMs), at Supercomputing 2026 (SC26) held in Chicago.

TIDE is a technology that continuously improves an auxiliary model by using data generated during actual AI inference. It is designed to optimize performance by adapting the inference system even when user requests and task types change.

The core is a technology called speculative decoding. When a small auxiliary model predicts multiple tokens in advance, a large model verifies them in a batch to speed up response generation.

Existing speculative decoding had a limitation in that performance fell when prediction accuracy dropped if actual user requests differed from the auxiliary model's learning patterns. TIDE improved this by progressively training the auxiliary model by reusing internal information generated by the large model during inference.

It also analyzes the auxiliary model's prediction accuracy and performance improvement effects in real time and automatically decides whether to use speculative decoding and whether to conduct additional training. It is designed to respond to changing workloads while reducing unnecessary computation.

According to research results More presented at SC26, TIDE increased AI inference throughput by up to 1.66 times compared with the existing method that does not apply speculative decoding. It also cut auxiliary model training time by up to 3.02 times versus the existing online learning method by reusing internal information from the inference process without separately generating training data.

It also confirmed performance improvements using heterogeneous GPUs. In an environment running the large model on Nvidia H100 GPUs and training the auxiliary model on AMD Instinct MI250 GPUs, overall system throughput increased by up to 1.22 times compared with putting the same GPU resources entirely into inference.

The study was conducted based on technology accumulated during development of More's AI inference platform, the MoAI Inference Framework. More is also focusing on developing infrastructure software called the Token Factory Operating System (OS) to efficiently allocate computation across different accelerators, including Nvidia and AMD GPUs and Tenstorrent AI chips.

More CEO Kangwon Cho (조강원) said, "AI infrastructure competitiveness depends not only on securing high-performance GPUs but also on how much AI computation can be processed with given computing resources." He added, "We will continue to develop software technologies that improve AI inference performance and resource utilization efficiency across diverse GPU environments."

Keyword

#Supercomputing 2026 #TIDE #Speculative Decoding #Nvidia H100 #AMD Instinct MI250
Copyright © DigitalToday. All rights reserved. Unauthorized reproduction and redistribution are prohibited.