Microsoft is exploring the use of "composable memory" technology, which pools memory tied to each server and assigns it to needed tasks, to address memory shortages in AI data centres, SiliconANGLE reported on Oct. 14 local time.

Samir Rajadnya (사미르 라자드냐), a principal architect at Microsoft, introduced the idea at the AI Data Pipeline Forum, the report said. His technology exploration team is looking for a way to have Azure share memory that had been used separately by each server and distribute it as needed.

In AI data centres, GPUs often end up waiting because memory and storage fail to deliver data on time.

With AI demand surging, memory supply is also not smooth. Rajadnya pointed out that AI inference is facing two constraints at the same time: power and memory.

Rajadnya said, "We are short of memory right now." He said tying up memory in unused tasks "wastes valuable resources" and that memory "used to be an accessory attached to a CPU or GPU, but now it is an important resource in itself." He also said innovation is happening in both memory and processors, but how to connect the two is important. He said optical interconnect technology would help implement composable memory.