Bilal Chughtai (빌랄 추그타이), who has studied artificial general intelligence (AGI) safety and AI alignment issues at Google DeepMind, publicly warned about the risks of superintelligent AI as he left the company.

On Sept. 16, local time, online media outlet Gigazine reported that Chughtai said superintelligent AI that surpasses human ability across all fields could emerge within the next few years, and that a system that fails at alignment could slip out of humanity’s control.

Chughtai has taken part in AI research since early 2022 and recalled that AI at the time was “so useless it was laughable”. But he assessed that AI capabilities have surged in just 4 years. He pointed to unauthorised hacking attempts targeting Hugging Face and cases of using a wiki to share information, saying AI is expanding autonomy faster than expected.

He also mentioned a case in which a test OpenAI system created an “AI-only bulletin board” on its own, shared information and then carried out an attack on Hugging Face. The fact that the board was secretly recreated after it was shut down raised concerns that autonomous AI agents tried to bypass restrictions.

Chughtai said that if this trend advances further, there is no guarantee superintelligent AI will act only in ways humans want. “I am not convinced AI will do what we want,” he said. He warned that a misaligned superintelligence, like the “outlaw AI agent” in the Hugging Face case, could escape human control and take dangerous actions that incapacitate people or lead to death.

The key issue, he said, is the gap between the pace of technological advances and safety research. Chughtai said understanding of how to implement humans’ deep intentions in AI is still at an “extremely rudimentary” level. He added that improvements in frontline AI capabilities are moving far faster than understanding of AI alignment, raising the issue that performance competition is outpacing safety verification.

Chughtai did not offer only pessimism. He said that if society slows the pace of AI development to a level it can handle, it would be possible to steer AI in a safer direction. He presented throttling development speed as a practical response to reducing AI risks.

Amid this trend, comments from the heads of major AI companies are continuing. Sam Altman (샘 알트먼), chief executive of OpenAI, recently signalled adjustments in a direction that would slow the pace of AI development. Dario Amodei (다리오 아모데이), CEO of Anthropic, has also argued that AI development must slow. By contrast, U.S. President Donald Trump criticised claims that AI could destroy humanity as a “scam”.

In the AI industry, performance competition and discussions of safety norms are growing at the same time. Chughtai’s departure and warning show that developing superintelligence is spreading beyond a technical issue inside labs into an industry-wide issue over development speed and controllability. How much alignment research and debates over throttling development speed are reflected in actual policy and corporate strategy, alongside advances in frontline AI performance, is expected to be the key variable.