A study found that about 1 in 6 U.S. adults could comply with voice phishing calls made with AI-generated voices. It found people were particularly vulnerable to scams posing as a family emergency, and that the persuasiveness of the dialogue mattered more than how realistic the AI voice sounded.

On Oct. 28 local time, Japanese media outlet ITmedia reported that researchers affiliated with Harvard University, Meta and other organisations conducted an AI voice phishing experiment involving 4,100 U.S. adults. The researchers compared six AI voice models — Meta's Llama Full Duplex, OpenAI AVM, Google Gemini, Sesame, PlayAI and ElevenLabs — with real human voices.

The experiment used five scenarios, including impersonating a card issuer and Gmail customer service, requesting donations, posing as police to demand bail for a relative, and asking for an urgent transfer while pretending to be a sister in danger. Participants reviewed call recordings or dialogue transcripts and then assessed their willingness to comply with the request.

Across all scenarios, an average of 16.5 percent said they would comply or found it difficult to decide. In a case that used ElevenLabs' voice-cloning technology to pose as a sister in danger, the response rate reached 36.1 percent.

Tactics appealing to emotion and personal relationships were statistically more likely to prompt compliance than typical customer service impersonation. The researchers analysed that "the persuasiveness of a story that triggers urgency and sympathy is a key factor, rather than the naturalness of an AI voice."

Some 70.3 percent of participants correctly identified an AI voice, but 67.4 percent mistakenly judged a real human voice as AI. Frequent use of generative AI or voice assistants did not show a clear difference in detection ability.

In an economic analysis, the study estimated that using U.S. labour costs as a benchmark, a human making scam calls would incur losses of about $27 per hour, or roughly 36,200 won. By contrast, using some AI models could generate expected profits of $1 to $3 per hour, or about 1,340 to 4,030 won.

The study measured survey responses rather than actual fraud losses, and 16.5 percent should not be interpreted as a real voice phishing success rate. The researchers warned that as AI voice technology spreads, scam calls could be automated and the cost of mass dialling could fall.