A new study has found that popular chatbots, including ChatGPT and Gemini, display different personality traits depending on the language used to interact with them. This means that changing the language of a question may produce a different answer, rather than simply a translation of the same response.
اضافة اعلان
Users may expect their favourite chatbot to provide the same answers regardless of the language they use. However, a new study published in the scientific journal Royal Society Open Science, and reported by Tech Xplore, suggests otherwise.
Researchers from the University of Science and Technology of China found that popular AI-powered chatbots display different personality traits depending on the language used to phrase questions and instructions.
Personality tests reveal differences in chatbot behaviour
The researchers subjected five chatbots to the Big Five personality test, using the second version of the personality assessment questionnaire in multiple languages.
The study included GPT-4o, Gemini 2.0 Flash, DeepSeek V3, ERNIE 3.5 and Lou Chat.
Each model answered a 60-item questionnaire in English and Mandarin Chinese, while Lou Chat was also tested in French.
The researchers conducted 30 trials for each language and found that the chatbots generally produced stable, positive self-descriptions. Their responses emphasised traits such as cooperativeness and dependability, while showing very low levels of negative emotionality.
However, the results revealed measurable differences in personality trait scores depending on the language used.
For example, GPT-4o scored 4.74 out of 5 for openness when tested in English, while its score rose to 4.81 when the test was conducted in Mandarin Chinese.
DeepSeek V3 showed the opposite trend, scoring 4.83 in English compared with 4.81 in Mandarin Chinese.
Gemini 2.0 Flash also produced different results, recording higher levels of conscientiousness a trait associated with discipline and responsibility alongside lower levels of negative emotionality when tested in Mandarin Chinese.
The research team compared the personality traits displayed by the chatbots with the results of human personality surveys conducted in the United States, China and France.
The comparison showed that none of the models fully matched the dominant personality traits of the human reference group in the country associated with the language used.
Another study reports similar findings
The new findings are consistent with previously reported observations by Anthropic, the company behind the Claude chatbot.
Earlier this year, the company said Claude’s behaviour varies depending on the language used in a conversation and the model version selected by the user.
Anthropic’s study analysed Claude’s behaviour during real conversations, while the new research used a standardised questionnaire to measure personality traits across several competing models.
The similar findings suggest that these differences may represent a recurring pattern in AI model behaviour rather than an isolated result from a single experiment.
Nevertheless, researchers in both studies were cautious about interpreting the findings.
The new study measures how models respond to questions and instructions, but it does not establish that chatbots possess genuine personalities in the human sense.
The researchers were also unable to conclusively separate the effect of language from other factors, such as training data, model design and the way statements are interpreted after being translated into different languages.
Anthropic has acknowledged that it does not yet know whether these differences represent a flaw in AI model performance or reflect the models’ ability to adapt to different cultural norms associated with each language.
Do you get different answers when you change languages?
For multilingual users, the findings have an important practical implication: asking the same question in another language may produce a different answer, rather than simply a translated version of the original response.
This means that using a second language to obtain another perspective from a chatbot may reveal differences in how it responds, even when the subject remains the same.
The findings highlight the role of language in shaping AI system responses and raise questions about how consistently these systems behave across different linguistic and cultural environments.
Source: Al-Ghad.
