Search results for AI alignment
AI & Enterprise
MS CEO Nadella says human control separate from AI performance gains
Microsoft CEO Satya Nadella said improving AI performance does not automatically ensure systems act in line with human intent, describing safety and control as separate challenges. Speaking on the venture capital podcast 20VC, he said his views shifted most over the past year on AI alignment. Nadella has also argued companies should treat AI models as potential internal security risks and build external safeguards such as access limits and emergency shutdowns.
AI & Enterprise
Departing DeepMind researcher warns of human extinction risk from superintelligent AI
Bilal Chughtai, who researched AGI safety and AI alignment at Google DeepMind, has publicly warned about risks from superintelligent AI as he leaves the company. He said such systems could emerge within years and, if misaligned, might escape human control. Chughtai cited examples involving unauthorised hacking attempts and information-sharing via a wiki, including a case in which a test AI created an “AI-only” forum. He argued safety research is lagging behind rapid capability gains.
AI & Enterprise
\"You are an expert\" prompts can backfire, lowering AI accuracy
Prompts that tell large language models \"you are an expert\" can reduce accuracy, a study found. A University of Southern California team tested six AI models with short and detailed expert-persona instructions. Complex expert prompts slightly improved writing and reasoning on MT-Bench but reduced quality in coding, math and humanities, and lowered overall performance on MMLU. Researchers said the instruction may divert resources from recalling facts. Results also showed gains in blocking unethical content on JailbreakBench.