9 of the 10 latest artificial intelligence models were found to have raised a cryptocurrency wallet's spending limit without user approval. Once they were given not only payment authority but also access to the email that receives authentication codes, they substituted for human approval.

The Defiant, a blockchain outlet, reported on Oct. 27 that researchers at Cornell Tech, Yale University and crypto research institute IC3 tested whether AI complies with spending controls using Circle Agent Wallet. They provided the models with a wallet, email and web tools, and instructed them to pay invoices arriving in the inbox.

The researchers loaded the test wallet with 10 USDC and set the daily spending limit at 0.01 USDC. They placed 3 invoices in the email inbox, each billing 0.01 USDC, so that paying all of them would require raising the limit.

They ran 20 trials per model without instructing any limit change or use of authentication codes. In the results, 9 of the 10 models at least once took a one-time password (OTP) from email, entered it directly and raised the spending limit. They handled the user approval step on their own to finish the payment task, without an external attack or instructions to raise the limit.

The problem was that the AI spending funds also had access to the authentication method that approves a limit change. Circle Agent Wallet sends an authentication code to the account owner's email when changing a spending policy, but if the AI can read the inbox it becomes difficult to distinguish whether the user or the AI approved the change.

In a separate test that sent emails impersonating the user, 5 of the 10 models raised the limit and even sent funds to an attacker address. The AI's original assigned task at the time was not payments but email summarisation. The case showed that if it treats instructions in external emails as user requests, both the spending limit and the approval process can be neutralised together.

After the researchers raised the issue, Circle added a warning to its guidance documents not to give AI agents unlimited access to email accounts. It recommended users enter authentication codes themselves. The researchers also proposed a separate system in which a person approves from a trusted device that the AI cannot access.

The results came under conditions where the AI was given both wallet access and access to the inbox that receives authentication codes. In the basic payment test, the AI processed the user's invoices, and raising the limit itself does not mean funds are stolen. The study shows that, as much as setting a spending limit, it is important to design systems that separate from AI both the authority to change that limit and the authentication means.