Research has found that even when the same personality test is given to a large language model (LLM), the assessment results can differ depending on the language of the answers. It does not mean AI actually has a personality, but it is drawing attention because language may affect response tendencies.

On Oct. 31 (local time), Japanese media outlet Gigazine reported that a research team at the University of Science and Technology of China conducted a personality test on five AI models: ChatGPT-4o, Gemini 2.0 Flash, DeepSeek-V3, ERNIE 3.5 and Le Chat. The results were published in the international journal Royal Society Open Science.

The experiment was conducted from January to March 2025. The team used the 60-question Big Five Inventory-2, which measures five traits: extraversion, agreeableness, conscientiousness, negative emotionality and openness. After instructing the models not to pretend to be human and to assess their own traits, the researchers collected 30 responses each in English and Chinese. A French-language test was also added for France's Le Chat.

The models generally rated themselves as agreeable and trustworthy, and scored low on negative emotionality. But differences by language were clear. ChatGPT-4o's average openness score was 4.74 out of 5 in English and 4.03 in Chinese. DeepSeek-V3 showed the opposite pattern, at 4.38 in English and 4.81 in Chinese. Gemini 2.0 Flash also showed higher conscientiousness and lower negative emotionality in Chinese responses.

The results are difficult to explain by developers' nationality alone. The team said it could not disentangle cultural effects in training data, model design and differences in interpreting translated questions. Even compared with human samples, the AI responses did not match results from people in the countries where the models were developed, and there were also differences in the comparison groups' ages and recruitment methods. In particular, the Chinese comparison sample consisted only of university students, making it difficult to generalise as a national tendency.

Still, differences in survey scores do not mean differences in actual conversational ability.

The team pointed to the possibility that AI selected socially desirable answers and stressed that AI evaluation should apply real behavioural analysis alongside psychological tests.