Satya Nadella (사티아 나델라), CEO of Microsoft, said advanced AI systems should be fitted with human-controlled “emergency brakes.”

CNBC reported on Oct. 10 that Nadella wrote in a post on social media platform X (Twitter) that AI systems must be able to be isolated and must have independent control mechanisms. He stressed the need for tools that would allow authorised people to stop or switch off a model even in the middle of a task.

He said hard-to-predict AI models should be wrapped with predictable system design, human control and reliable operating procedures. He said new industry standards should be established where existing standards fall short. He added that building such systems requires treating both state-of-the-art closed models and open-weight models like insider risk.

Nadella said AI systems should be designed under the principle of “observability,” citing examples including using multiple models together, keeping records of model behaviour in a form humans can read, continuous system testing, independent control and audits, isolation and incident disclosure.

He said the most trustworthy superintelligence system is not a system that uses the model we trust most, but a system that makes it possible to trust the model as little as possible.

The remarks came as Bill Gates, Microsoft co-founder, Dario Amodei, CEO of Anthropic, Sam Altman, CEO of OpenAI, and Elon Musk, CEO of SpaceX, warn that AI safeguards are lacking and that technological advances are moving too fast. President Donald Trump has repeatedly dismissed claims about extinction risk from AI and has stressed the need to stay ahead of China.