Korea-made on-device artificial intelligence (AI) semiconductors have reached global-level hardware performance but are failing to clear the commercialisation hurdle because of software barriers. Neural processing units (NPUs) developed by domestic fabless companies and AI startups have proven competitive in compute specifications and low-power efficiency, but cases are continuing in which they drop out at the proof-of-concept (PoC) stage because the software development kits (SDKs) and compiler quality needed for customers to run the chips are lacking.

On July 21, the industry said set makers in home appliances, mobility and robotics often give up on adopting Korean AI chips even if they consider them at the PoC stage because the development environment is not in place. The reason is a lack of tools and development frameworks that allow customers to optimise (compile) and deploy their own AI models on the chip hardware. For set makers' developers, that leads to the calculation that returning to foreign chips based on Nvidia's CUDA costs less in development than grappling with an unfamiliar development environment for Korean NPUs. As a result, Korean chips lose even the chance to show their performance.

This bottleneck is not unrelated to the investment structure of domestic fabless companies. With limited budgets and manpower focused on physical chip design, software development such as compilers and SDKs has been pushed down the priority list. Even though they succeeded in raising tera operations per second (TOPS), a hardware-spec metric, they have fallen short in responding to the need to connect it to customers' development environments.

Technically, problems concentrate in the compilation process. When AI models written in PyTorch or Open Neural Network Exchange (ONNX) format are converted into low-power machine language for NPUs, unsupported operators can cause those operations to shift to the CPU, creating a bottleneck. It is also reported there are cases in which set makers abandon mass-production deployment because accuracy loss is large in the quantisation process that compresses 16-bit floating-point (FP16) models to 8-bit integer (INT8). Difficulty in tracing the cause even when mapping fails or performance drops is another factor blocking adoption.

A lack of debugging tools to find errors is also cited as an obstacle. Without a profiler like Nvidia Nsight or Apple's Core ML SDK that visually shows which layer creates a bottleneck, set makers' developers must manually find errors one by one.

As development burdens grow, voices in the field at set makers are complaining of fatigue from the verification process for Korean chips. While AI model trends such as large language models (LLMs), vision transformers and small language models (SLMs) change every month, the speed of SDK updates by domestic NPU startups cannot keep up, leaving set makers feeling it is burdensome to use Korean chips.

Research institutions are also making similar on-the-ground diagnoses. In a report on AI semiconductor competitiveness from a software perspective, the Electronics and Telecommunications Research Institute (ETRI) said the AI semiconductor market is structured so that individual compiler development is required for as many combinations as there are deep-learning frameworks and chip types. It warned that if domestic fabless companies short of software personnel cannot build a general-purpose compiler and framework ecosystem, then no matter how high NPU performance is, they could fall into a state where mass production and sales are impossible.

The National Information Society Agency (NIA) and the Science and Technology Policy Institute (STEPI) also pointed to technical compatibility at the PoC stage and the absence of SDKs as factors behind the failure of Korean AI semiconductors to secure meaningful demand. They analysed that the success or failure of Korean NPUs depends not on the compute specifications of a single chip but on the software ecosystem and on securing validation references from set makers and data centres.

AI infrastructure company Modular pointed to a combinatorial explosion, or balance problem, as a reason hardware companies fail to build AI software. Each time hardware technology such as semiconductors advances, software teams must support every combination of AI model types, quantisation methods, batch sizes and frameworks. Software teams at fabless companies, short of engineers by the hundreds to thousands, are preoccupied with responding to immediate customer errors and cannot reach the stage of building a long-term compiler ecosystem, it said.

For example, even Nvidia, which has global dominance, is assessed as deriving its strength not from GPU compute speed but from a CUDA document, SDK and custom-kernel ecosystem rooted over the past 15 to 20 years in universities and research institutes around the world. For developers and companies, learning a new NPU SDK beyond CUDA itself is a massive opportunity cost. An industry official said, "Since it is difficult for individual fabless companies to build an ecosystem comparable to CUDA on their own, there is a need for a platform that can distribute the development burden and that chip suppliers and set makers can use together."