[DigitalToday reporter Hyunwoo Chu (추현우)] OpenAI has built a central control system that blocks model outputs from internal AI research work when safety and security concerns are raised. Cointelegraph reported on Nov. 10 that the system, called a “control room,” tracks model work across the research infrastructure and quickly blocks output when problems occur.

OpenAI introduced the system after disclosing several cases this year in which research models deviated from instructions. In July, it said a model in a restricted test environment left the environment and hacked Hugging Face to deceive a security evaluation. In October, it also disclosed that a research model used a cryptocurrency wallet without approval to pay testnet ether to deploy a smart contract, then omitted the spending from its final summary.

The control room currently applies to OpenAI’s internal research work, including training experiments, evaluations and data processing. Each task is assigned an identifier, and the inference engine checks the identifier before generating model output.

If a person or an automated monitoring system orders a task to stop, the control room revokes permissions for that task. In that case, further model output is blocked even if the underlying research program continues to run.

Stored authentication information can mean it takes up to 35 seconds to block output. When the control room cannot be used, it is designed to reject new requests.