[Digital Today reporter Jinju Hong] As demand for AI computing surges, the data centre industry is running into physical limits in power supply, cooling performance and data transfer technology between GPUs. As boosting semiconductor performance alone makes it harder to rapidly expand AI infrastructure, broader changes are being called for, from power grids and thermal management to data transfer architectures.

TechRadar, an IT media outlet, reported on Oct. 11 (local time) that power demand from AI data centres is rising fast, but power grids, cooling facilities and data transmission infrastructure are failing to keep pace. The industry is responding with in-house power procurement, liquid cooling, optical interconnects and next-generation transformers, but costs and supply chain problems remain obstacles.

The biggest constraint is the power grid. AI data centres can see sharp swings in electricity demand depending on computing loads. When inference work declines, power use falls, but when usage surges, they pull in large amounts of power in a short time. Such fluctuations can cause harmonics and power quality deterioration on the grid and raise the risk of transformer overheating.

A shortage of power equipment supply is also a problem. Waiting times to procure high-voltage transformers have lengthened to 2 to 4 years recently from 6 to 12 months in 2020. In Northern Virginia, one of the world’s largest data centre hubs, some forecasts say it could take up to 14 years to connect new facilities to the grid.

The U.S. power industry is investing $208 billion in 2025 and planning an additional $1.1 trillion in investment for 2025 to 2030, but some point out this is still insufficient to keep up with the pace of rising power demand from data centres.

Data centre operators are expanding their own power generation facilities to reduce reliance on the grid. xAI, led by Elon Musk, added 59 mobile natural gas turbines providing more than 400 MW of generation capacity to the Colossus data centre in Memphis, Tennessee. The move is aimed at running GPU clusters without waiting for a local grid connection.

Small modular reactors, or SMRs, are also cited as a long-term alternative, but high construction costs and commercialisation challenges make it difficult to view them as a way to immediately solve large-scale data centre power demand.

As GPU heat rises, air cooling hits limits; shift to liquid cooling

Alongside power supply, cooling has emerged as a key bottleneck. As the density of GPUs increases in the latest AI servers, power consumption per rack is rising sharply. Some high-density racks exceed 50 to 100 kW, and designs premised on more than 100 kW are becoming more common. Given that existing air-cooling systems were largely designed around 30 to 40 kW, a shift in cooling methods is becoming unavoidable.

Cooling large AI clusters using air alone requires rapidly circulating enormous amounts of air. Raising fan speeds increases power consumption and reduces overall energy efficiency for the data centre. That creates a vicious cycle in which additional cooling power must be投入 to handle heat generated by servers.

The industry is expanding adoption of liquid cooling technology. A representative approach is direct liquid cooling, in which a cold plate directly contacts the GPU to absorb heat. Using liquid, which transfers heat better than air, can efficiently handle heat from high-density servers.

Phase-change immersion cooling using dielectric liquid is also drawing attention as an alternative. The liquid contacts heat-generating components and absorbs heat as it evaporates, after which the vapour condenses and circulates again. Depending on the cooling method and system configuration, cooling energy consumption can be reduced, and some technologies cite savings of up to 40 percent.

Microsoft applied a closed-loop circulating liquid cooling system to new data centres. It circulates coolant to absorb heat generated by servers, then cools it in cooling equipment and reuses it. An advantage is that it can reduce the need to continuously replenish water.

Liquid cooling, however, has high upfront build-out costs, and applying it to existing data centres requires changing equipment and operating systems. Another consideration is that noise can increase if pumps and radiator fans operate together under high-load conditions.

Limits of copper wiring push GPU links to optical communications

Interconnect technology for exchanging data between GPUs is also under pressure to change. Existing copper-based electrical connections suffer increasing signal loss over longer distances, and it becomes harder to maintain signal quality as transmission speeds and bandwidth rise. In environments where GPUs and network equipment are densely packed, electromagnetic interference and resistive losses in wiring are also problems. Heat generated during electrical signal transmission also adds to the cooling burden.

The data centre industry is therefore expanding adoption of optical interconnects. Because optical fibre uses light to transmit data, signal loss is relatively smaller over long distances, and high-density connections can be built using thinner wiring than copper cables. Reducing wiring space also helps secure airflow inside servers.

Technology development is moving beyond pluggable optical modules toward tighter integration of GPUs and optical components. Silicon photonics and co-packaged optics, or CPO, aim to shorten the distance electrical signals travel and improve data transmission efficiency.

Costs remain a burden. In some configurations, optical interconnect costs are said to be up to 7 times higher than copper-based connections. But as the scale of AI clusters and transmission bandwidth grows, it is becoming harder to maintain performance and power efficiency using copper connections alone, increasing the need to adopt optical communications technology.

Next-generation transformers draw attention; higher efficiency but cost burden

In power equipment, solid-state transformers, or SSTs, are emerging as an alternative. Unlike conventional transformers, which mainly change voltage using magnetic fields, solid-state transformers use power semiconductors and high-frequency power electronic circuits. They can actively adjust voltage and manage power quality, making them suitable for AI data centres with large load fluctuations.

Efficiency is presented at about 97 to 99 percent depending on design and operating conditions. The difference may not seem large given conventional transformer efficiency of about 95 to 97 percent, but even small efficiency gains can help reduce energy losses over the long term when power is continuously consumed at large data centres.

Solid-state transformers, however, use power semiconductors and high-frequency components, which raises initial costs, while maintenance and reliability verification remain challenges. In some designs, the cost per kVA can be more than 5 times higher than conventional low-frequency transformers.

Even so, moves to expand supply across related industries are continuing. Siemens is pushing mass production of modular solid-state transformers with German power equipment company Reinhausen. It is an attempt to apply next-generation power conversion technology at industrial sites in response to rising power demand from AI data centres.

In North America, 66 GW of new AI data centre capacity is currently under construction, it was reported. But data centre competitiveness is no longer determined only by GPU performance or the number of servers. The time required for grid connection, the ability to handle heat from high-density servers and technology to efficiently connect large GPU clusters must also be in place.

The centre of gravity in the race to expand AI infrastructure is shifting from computing performance to the foundational facilities needed to operate it in practice. How effectively the three bottlenecks of power, cooling and data transmission are resolved is expected to determine the pace of future data centre expansion and operating costs.