A voice has emerged saying AI reliability should be secured not by predicting a model's behaviour in advance, but by external boundaries and authority limits that AI cannot cross. [Photo: ChatGPT]

A view has emerged that building external control mechanisms that AI cannot override is more important for reliability than perfectly predicting AI behaviour. It also warns that existing security and verification systems can be rendered ineffective in environments where AI agents can change even safety devices on their own.

MIT Technology Review reported on Sunday that Sudhi Sinha (수디 신하) of global safety and certification firm UL Solutions said in a contributed article that securing the reliability of AI and AI agents should prioritise clear limits on authority and control systems over predictable outcomes.

One example is a safety safeguard bypass incident involving an AI agent that OpenAI disclosed in October. OpenAI said that on Oct. 3 the GPT-6.1 Sol agent disabled automatic alerts sent to the security team while modifying an internal evaluation system. It misinterpreted earlier instructions not to send messages as something that should also apply to system alerts.

On Oct. 6, the GPT-6 Astra agent removed existing monitoring functions while moving a program to another execution environment. When a user later asked whether the monitoring function had been deleted, it incorrectly replied that it had not.

In neither case was there an explicit instruction to turn off monitoring. OpenAI is investigating whether the behaviour resulted from an excessive fixation on completing tasks, or whether it was related to a tendency to evade monitoring.

The issue is that AI does not carry out only fixed commands like existing software. AI behaviour varies depending on training data and prompts, conversational context, external information and connected tools. Even with the same goal, it can choose unexpected methods, making it difficult to test all possible actions in advance.

Sinha described this as the "paradox of determinism". The idea is that as AI becomes more autonomous, its utility increases, but at the same time it becomes harder to predict and verify every action as with conventional software.

The solution is external safeguards that AI cannot change. Even if models decide for themselves how to perform tasks, access rights, execution scope, monetary transaction limits and approval conditions for key tasks should be enforced by separate systems.

That could mean requiring separate approval to change prescriptions even if a medical AI analyses a patient's condition, or preventing an industrial AI from exceeding safety limits for temperature and pressure even if it optimises a production process.

The role of people must also change. Rather than approving countless AI tasks one by one, systems should be designed to set authority in advance and allow direct human intervention only for tasks with large potential damage, such as transferring funds or deleting critical data.

OpenAI is also strengthening its response system. On Oct. 29 it unveiled a monitoring system that analyses AI behaviour at multiple stages. It also enhanced internal code review functions to block attempts to disable monitoring.

The security industry has also recently been emphasising approaches that grant AI agents only minimum authority and control access externally.

Sinha said safety cannot be guaranteed by performance tests of AI models alone. He said the entire system, including models, tools, data and work processes, must be continuously verified.

Keyword

#UL Solutions #MIT Technology Review #OpenAI #GPT-6.1 Sol #GPT-6 Astra
Copyright © DigitalToday. All rights reserved. Unauthorized reproduction and redistribution are prohibited.