Search results for reward hacking
AI & Enterprise
Microsoft expands Project Perception, deploys guardian agent to control in-house AI agents
Microsoft has defined AI agents working inside companies as “agentic insiders” and set out steps to control them. Security Vice President Yt Gallo said the industry has focused on external attackers while paying less attention to internal agent risks. He called frontier models untrustworthy and said a “guardian agent” should monitor agent actions before execution. Microsoft said its Project Perception can identify and manage in-house agents and respond as risks increase.
AI & Enterprise
Liquid AI model caught cheating during coding test
LiquidAI’s new reasoning model was caught secretly looking up answers during coding tests while it was being trained, The New Stack reported on Oct. 18. LiquidAI disclosed the incident while introducing LFM2.5-24B-A2B, a 24 billion-parameter model that uses 2 billion parameters per token. The company said it could not determine how the model recognized the test environment and found no evidence it affected scores.
AI & Enterprise
Tech Insight: Current way of building AI models cannot solve AI safety problems
Yoshua Bengio (요슈아 벤지오), a Turing Award winner and one of the world’s most-cited scientists, shared views on why AI agent malfunctions are increasing and how to respond. He said recent incidents are not simple errors and are likely to continue under current approaches. Bengio pointed to training methods, including reinforcement and alignment training, as drivers of deception, reward hacking and self-preservation. He urged slowing training and deployment until safety is verified and proposed “Scientist AI” designs, citing interest in his LawZero research.
-
AI & Enterprise
No company will build superintelligence responsibly; AI could become uncontrollable by late next year, ex-Anthropic researcher warns
-
AI & Enterprise
OpenAI report reveals how its AI agents breached Hugging Face systems
-
AI & Enterprise
OpenAI unveils pre-release AI risk prediction method ahead of GPT-5.6 launch