SK hynix has set out a technology to address a new bottleneck in AI infrastructure in an international academic journal. SK hynix said on Wednesday it published a paper in Nature Electronics with global researchers on the development path for next-generation optical interconnect technology, known as CPO (Co-Packaged Optics).

The company said HBM resolves memory bottlenecks inside the package of AI accelerators. But as ultra-large clusters linking thousands of GPUs and HBM emerge, a "bandwidth barrier" limiting data movement between systems has become a new bottleneck. Computing performance is rising about threefold every two years, while bandwidth is increasing only 1.4 times, widening the gap.

The research was led by corresponding authors Seung-hoon Hong (홍승훈), a team leader at SK hynix, and Gyo-sang Lee (이규상), a professor in the Department of Electrical and Computer Engineering at the University of Virginia, who oversaw the entire process. Researchers from the University of Illinois Urbana-Champaign, Nanyang Technological University, the Massachusetts Institute of Technology and Yonsei University also took part. The paper is significant for systematising a comprehensive technology roadmap for how memory, packaging and optical interconnects should converge.

Lee said, "Even if computing chip performance improves, overall system performance does not rise if the ability to move data between chips cannot keep up." He said, "Replacing copper wiring, which has physical limits, with light to move data is the most viable scaling path." Hong described CPO as "a technology that puts optical transceivers in the same package as the processor so chips can exchange data using light instead of long electrical wiring."

In the paper, the researchers set technical targets for next-generation AI infrastructure, including more than 100 Tb/s of bandwidth per node, energy consumption of less than 1 pJ/bit and inter-chip latency of less than 10 ns. They completed a roadmap by organising the technology path from 2D packaging to 2.5D interposers and 3D heterogeneous integration, along with key tasks for commercialisation. Over the long term, the concept is to maximise data-movement efficiency with an "optics-centric" architecture that directly connects memory and processors through optical interposers.

Lee said, "Optical interconnects are likely to become a core connectivity technology for AI infrastructure." He assessed that "the related technology has already moved beyond the lab and entered the early stage of industrialisation." He added, "Ultimately, the key is to co-design memory devices and controllers, optical devices and packaging as a single system," and said, "Cooperation with SK hynix is meaningful at precisely this point."

Hong said, "It is meaningful that we linked academic expertise and on-site industrial experience to present the future direction of AI infrastructure together." He said, "We will continue to expand open cooperation that can provide tangible value to customers and the industry ecosystem."