Mustafa Suleyman (무스타파 슐레이만), CEO of Microsoft’s AI group, said artificial intelligence development is unlikely to run into limits for the time being.

In an article contributed to MIT Technology Review on April 8 local time, Suleyman pointed to an explosive rise in computing power as a key driver of recent AI performance gains. He said repeated claims of a growth slowdown, from inside and outside the industry, have proven wrong.

He said the amount of computation used to train cutting-edge AI models has expanded since 2010 from 10 to the power of 14 flops to more than 10 to the power of 26. He argued the rise has been a key factor driving overall AI advances.

He also pushed back against claims that the slowing of Moore's Law, a lack of data and power constraints explain the current improvement trend. Recent gains are the result of not only semiconductors but also simultaneous improvements in memory, networks and software efficiency, he said.

There have also been major changes on the hardware side. According to Suleyman, the raw performance of Nvidia chips rose about eightfold over six years, from 312 teraflops in 2020 to 2,500 teraflops recently. Microsoft’s in-house AI chip, Maia 200, was also introduced as delivering about a 30 percent improvement in performance per dollar versus the previous version. He said connectivity technologies such as HBM, NVLink and InfiniBand made it possible to tie hundreds of thousands of GPUs together as if they were a single system.

These infrastructure improvements have shortened training times. Suleyman said it took 167 minutes in 2020 to train a language model using 8 GPUs, but it has now fallen to under 4 minutes. He said performance improved by about 50 times, far exceeding the roughly fivefold improvement expected under Moore's Law.

Software efficiency is also improving quickly. Research group Epoch AI said the amount of computation needed to achieve the same performance is halving about every eight months. The annualized service costs of some latest models have also been shown to have fallen by up to 900 times.

Forecasts also point to a faster pace of growth ahead. The capabilities of major AI research organisations are expanding about fourfold a year, and since 2020 the computation used to train leading models has been rising by about five times a year. Global AI computing is projected to reach the equivalent of 100 million H100s by 2027. Suleyman said that if the trend continues, effective computing capacity could expand by about another 1,000 times by 2028.

Suleyman stressed the changes will alter the very form of AI. "Forget basic assistants that answer questions," he said. He forecast that beyond simple question-and-answer tools, AI will develop into "human-level agents" that carry out long-term projects, negotiations and logistics management. He said this would affect industries across the cognitive-labour economy.

Power was cited as the key remaining constraint. One large AI rack the size of a refrigerator consumes about 120 kilowatts of electricity, he said, equivalent to around 100 households. He added that the cost of solar power has fallen to nearly 1/100th over 50 years and battery prices have dropped 97 percent over 30 years, and that an expansion path based on clean power is beginning to emerge.

Large-scale investment is also already becoming a reality. Suleyman stressed that an AI cluster worth $100 billion and a power demand of 10 gigawatts are no longer science fiction. He said related projects are under way in the United States and other regions, and that Microsoft is also designing a superintelligence lab on that premise.

He said that while AI scepticism will continue to be raised, the surge in computing power is "the technological story of our time". Bottlenecks in AI development exist, he said, but the expansion phase for computing infrastructure is still only at an early stage.