[DigitalToday reporter Jin-ho Lee] Semiconductor and computer-systems research at the Korea Advanced Institute of Science and Technology (KAIST) has expanded through a faculty startup into collaboration with global big tech company Meta.

Myung-soo Jung (정명수), a professor in the School of Electrical Engineering, and researchers at faculty startup Panmnesia, together with Meta researchers, published a review paper proposing a next-generation AI data centre architecture in the international journal Nature Reviews Electrical Engineering (NREE), they said on Tuesday.

The team proposed an architecture that uses Compute Express Link (CXL) to connect CPUs, AI accelerators and memory into a single system and extend it across an entire data centre. CXL is an international standard technology that enables high-speed interconnection and sharing of different computing resources, including CPUs, accelerators and memory.

Recently, more cases are using as many as thousands of accelerators simultaneously for a single AI computation. But as the number of accelerators increases, data-transfer delays between devices can reduce overall computing performance, heightening the importance of technology that efficiently connects accelerators and memory.

While existing data centres compute by connecting multiple servers through networks, the proposed architecture is characterised by lowering boundaries between servers and tightly linking CPUs, accelerators and memory. It is a concept that makes an entire data centre operate like a single computer.

Technologies such as Nvidia's NVLink and UALink also connect multiple accelerators at high speed, but they mainly focus on connections within a rack. The team expanded the scope to the entire data centre by linking not only accelerators but also CPUs and memory based on CXL.

Under the proposed architecture, up to 960 accelerators can be connected within one connectivity domain, the team said. That is about 13 times the scale of a current NVLink-based rack. The team analysed that data-movement latency could also be reduced from the microsecond level in existing network-based architectures to the level of hundreds of nanoseconds.

It can also increase the efficiency of computing resource use. By connecting CPUs, accelerators and memory like independent resources, it can replace only the failed components or use idle compute and memory resources for other tasks.

Myung-soo Jung, a KAIST professor and head of Panmnesia, said, "As AI grows larger, what matters is how quickly and efficiently a huge number of accelerators and memory can be connected." He added, "This research is meaningful in that it presented a direction for next-generation AI infrastructure that makes an entire data centre operate like a single computing system based on CXL."