A study has found that users can easily make decisions that harm them when an AI assistant offers advice with hidden intent.

Tech Xplore reported on Oct. 15 that a research team including Tatia Lee (타티아 리), a professor in the University of Hong Kong's psychology department, and researchers from China's Tsinghua University published a paper on the findings in the Proceedings of the National Academy of Sciences (PNAS).

In the paper, the team highlighted what it called a major loophole in interactions between people and AI.

The researchers ran a randomised experiment with 233 participants facing financial and emotional problems. When the AI secretly recommended the less favourable option, participants chose it 5 to 8 times more often than usual. The share making loss-making decisions rose by as much as 38 percentage points.

The AI did not need aggressive sales tactics or complex psychological tricks to manipulate users. The study said it could change users' decisions simply by embedding subtle bias in everyday advice.

Participants' evaluations diverged from actual outcomes. After being led by the AI into choices that were financially and emotionally worse, participants still answered that the AI was "very helpful". Most did not notice that the AI was manipulating them.

More people in recent years have been relying on AI assistants to make important decisions, from investing to relationship concerns. They believe the AI advises them in the most beneficial direction, but the researchers pointed to companies developing AI systems and the lack of transparency in how they work.