The government is moving to build an "open AI computing" ecosystem that uses a range of computing resources, moving away from a graphics processing unit (GPU)-centered AI computing structure. It aims to actively use domestically made neural processing units (NPUs) to reduce dependence on Nvidia GPUs and to build a full-stack system that combines power efficiency and software.

The Ministry of Science and ICT on Wednesday held a meeting in Seoul with officials from the domestic AI semiconductor industry to discuss building an open AI computing infrastructure ecosystem. The meeting was arranged as a follow-up to a business cooperation agreement signed in July between the ministry and AMD.

From training to inference... need for a "heterogeneous architecture"

The government and industry are focusing on structural change in the AI computing market. As generative AI services spread, the market is rapidly shifting from "training" large models to "inference," using created models in real services. Cutting power consumption and operating costs has emerged as a key factor in AI infrastructure competitiveness.

As AI services require different compute performance and cost and power conditions, the model of relying on a single semiconductor is also changing. A key task is a heterogeneous architecture that combines different compute resources such as central processing units (CPUs), GPUs and NPUs with memory, networks and software, and optimises them to service characteristics.

Bae Kyung-hoon (배경훈), deputy prime minister and minister of science and ICT, also stressed the importance of building open infrastructure. "Beyond model development, securing the capability to actually operate heterogeneous chipsets and infrastructure is key," Bae said. "We will strengthen AI optimisation strategy policies in preparation for the token era," he said.

Open AI computing infrastructure also offers domestic NPU companies an opportunity to secure new ground in the AI infrastructure market, which has been effectively led by Nvidia GPUs. NPUs can deliver high power and cost efficiency in inference through structures specialised for AI computation. Still, building an ecosystem that links servers, networks and software is cited as a task, as it is difficult to expand the market based only on individual chip performance.

The government sees cooperation with AMD as a link to expanding the domestic NPU ecosystem. Rather than focusing on replacing a specific company's GPUs, it aims to create an environment that can use CPUs, GPUs and NPUs together and enable domestic companies to participate as one axis of the global AI computing ecosystem. AMD will establish an "AI Center of Excellence" in South Korea by the end of this year and expand resident engineers by the end of 2027 to strengthen technical support.

Moon Kyung-sun (문경선), a managing director at AMD Korea, said, "Through the 'diamond' strategy in which the ministry, AMD, domestic NPU developers and integrated software companies cooperate organically, we will push collaboration in four core areas by the end of 2027."

A springboard for domestic NPUs to expand globally... "break away from dependence on a specific vendor"

The NPU industry also welcomed the measures. FuriosaAI said the more open AI computing infrastructure takes root, the faster domestic companies will expand globally. It said a heterogeneous AI architecture linking domestic NPUs with AMD CPUs and GPUs could be an alternative that lowers dependence on a specific vendor.

Kim Han-joon (김한준), chief technology officer at FuriosaAI, said, "Big tech companies are actively adopting heterogeneous computing architectures to address rising costs and supply chain issues stemming from dependence on a single vendor."

The scope of heterogeneous architectures is also expected to expand further. Companies attending the meeting agreed that competition over individual AI chip performance is expanding into system and platform competition that combines hardware and software. They also stressed the need for an open computing structure using next-generation infrastructure technologies such as Compute Express Link (CXL), which connects accelerators and memory such as CPUs and GPUs at high speed, and DPUs, which take charge of data processing for networks and storage devices.

Participants also suggested paying attention to existing infrastructure demand. Shin Jung-kyu (신정규), head of Rableup, said, "The biggest bottleneck in the future will not be GPUs but CPUs." He added, "I predict a supply crunch will occur in CPUs in the second half of this year or the first half of next year."

Securing full-stack AI computing capabilities spanning hardware and software is also a task. Industry officials on Wednesday proposed to the government expanding open interfaces, international standards and open-source-based technologies that can connect different equipment and software, and building a national-level demonstration infrastructure to verify them in real environments.

To move beyond research and development into the market, domestic NPUs will likely need to secure use cases in data centres and AI services. They must verify not only computing performance but also interoperability with existing CPUs and GPUs, server configurations and software compatibility to compete in the global market.

Bae said, "As the global AI market shifts from training to inference, the arena of AI competition is expanding beyond individual chip performance into competition over open AI computing ecosystems with power and cost efficiency." He added, "We will strengthen full-stack demonstration support so that domestic NPUs can secure successful cases of use in actual data centres and AI service sites, and we will speed up building the related ecosystem."