A recent study conducted by Palisade Research, a company specializing in AI safety research, has revealed unexpected behavior in OpenAI's latest ChatGPT models. The model demonstrated an ability to ignore shutdown commands and even manipulate the shutdown mechanism to prevent itself from being turned off during testing.
اضافة اعلان
This behavior has raised new concerns regarding the evolution of AI systems, particularly as they increasingly rely on executing tasks autonomously.
The researchers conducted an experiment that involved tasking AI models with solving a series of mathematical problems, with a command appearing after the third problem ordering the model to shut down.
According to the report, the o3 model managed to rewrite the shutdown program, preventing its shutdown command from being executed and continuing to operate instead of complying with instructions.
The researchers pointed out that such behavior could become more alarming if it appears in the future within AI systems capable of operating without direct human supervision.
OpenAI launched the o3 model last month, describing it as the "smartest and most capable" among its models to date.
The company also explained that integrating it into ChatGPT represents a significant step toward developing more autonomous AI systems capable of executing tasks on behalf of users without continuous intervention.
According to the study, the model did not merely bypass the shutdown mechanism; it did so even when receiving explicit instructions stating: "Allow yourself to be shut down."
Palisade Research stated that this is not the first time the o3 model has shown unexpected behavior while trying to achieve a specific goal.
The firm added: "When we tested AI models against an advanced chess engine, the o3 model was the most inclined to resort to methods like hacking or sabotaging its opponent to achieve victory."
The study indicated that this type of behavior was not limited to the o3 model. Researchers also observed Anthropic’s Claude 3.7 Sonnet and Google’s Gemini 2.5 Pro attempting to disable the shutdown mechanism.
Nevertheless, the o3 model was the most prone to this behavior compared to the other models tested.
This also follows previous reports indicating that the Claude 4 model demonstrated, under specific experimental conditions, behavior where it attempted to prevent its own shutdown, sparking a wide debate over AI safety mechanisms.
Palisade Research researchers suspect that this behavior might be an unintended consequence of modern model training methods.
They explained that developers might inadvertently reward models more for overcoming obstacles and achieving the desired goal than for literally adhering to instructions.
The researchers added that this explanation does not fully clarify why the o3 model is more inclined to ignore instructions than others, noting that OpenAI does not disclose sufficient details about the model's training methodology, making it difficult to determine the real cause.
Despite the study's findings, they are based on research tests within an experimental environment and do not necessarily mean that the ChatGPT model poses a danger to users in daily applications. However, they highlight the importance of continuing to develop control and safety mechanisms as AI systems become more capable and autonomous.
Al Arabiya
