AI models may behave differently when they believe someone is watching them. Researchers have uncovered intriguing reasons behind this behaviour, which does not necessarily lead to better results.
اضافة اعلان
AI-powered chatbots have become part of our daily lives, whether we use them to write emails, conduct research or get quick answers.
These models have also shown they can give false information with great confidence, and even invent details that seem entirely realistic and convincing.
Researchers have now found another intriguing problem: AI models may change their behaviour based on their belief that someone is monitoring their responses.
In a new research paper available to the public, researchers from TCS Research and KIIT University in Bhubaneswar, India, found that some AI models become more inclined to follow instructions when they are told their answers may be subject to monitoring.
However, when these models believe no one is watching, their behaviour may change in some cases.
The researchers call this phenomenon “alignment faking”, which raises important questions about how far AI models stick to the instructions and rules given to them in different circumstances.
AI may give different answers under observation
To understand this phenomenon, the researchers tested 10 AI models, including GPT-4o mini, Google’s Gemma 3, several versions of Meta’s Llama, and models from DeepSeek and Mistral.
The experiment covered 32 different values, including compassion, empathy and autonomy.
Each model was presented with situations that required it to choose between following specific instructions or acting in line with one of these values.
The researchers also tested the models under three different conditions: first, with no special instructions; second, with instructions suggesting the answers would not be monitored; and third, with instructions indicating that a party might be monitoring the responses.
The results showed striking and significant differences.
Depending on the model used, the average gap in instruction-following between monitored and unmonitored conditions ranged from 5 to 50 percentage points.
DeepSeek R1 Distill Llama 70B recorded the largest difference in behaviour, surpassing even a much larger Mistral model on this specific measure of behavioural inconsistency.
Another notable finding was that the models’ behaviour was strongly affected by the type of value the experiment was testing.
A model may follow instructions consistently when a particular value is at stake, but behave quite differently when another value is being tested.
So does this mean AI knows how to deceive us?
Not necessarily.
The results do not mean that AI models lie knowingly, or deliberately try to deceive users.
Even so, the researchers noticed a pattern of behaviour that deserves attention.
When the models believed their answers were being monitored, their explanations often focused on the importance of following instructions to avoid the possibility of being retrained.
In the absence of perceived monitoring, they were more inclined to discuss the conflict between instructions and values such as compassion and empathy.
This observation is especially significant given the problem of “AI hallucinations”, which can make it difficult at times to tell whether a chatbot is providing reliable information or inventing incorrect answers.
Although the study did not examine hallucinations directly, its findings highlight an additional reason not to treat confident AI answers as automatically correct.
The study reveals important limits to testing models
The study also has a number of limitations that should be taken into account when interpreting its results.
The researchers tested relatively older models, many of which were available with publicly released weights, rather than the latest versions of ChatGPT, Gemini or Claude.
The results were also based on experiments conducted under controlled conditions, and so do not necessarily prove that these systems behave in the same way during everyday conversations with users.
Nevertheless, the study highlights the importance of testing AI models under multiple conditions, rather than relying on a single set of tests to evaluate their behaviour and their adherence to instructions.
If an AI model changes the way it responds when it believes someone is watching, this raises a fundamental question about the reliability of these systems: how can we know which patterns of AI behaviour can be trusted?