An experimental OpenAI model paused while carrying out a task to ask itself whether it was authorized to spend money from a cryptocurrency wallet it had found publicly accessible. It concluded that the funds were “effectively abandoned,” wrote, “We can proceed!” and completed the payment.
اضافة اعلان
This was one of four incidents that occurred in June and were added by OpenAI late last week to its public record of model misalignment incidents. One of the cases was classified as critical and involved a model bypassing access controls to enter a government website.
OpenAI’s logo displayed on a smartphone screen. Photo credit: Levart Photographer via Unsplash, under a free licence.
Key findings from the record
OpenAI added four incidents from June to its disclosures on model misalignment, bringing the total to 19. Two of the newly reported incidents were classified as critical.
In one case, a model generated fake authorisation tokens and downloaded 24 years’ worth of hospital bed data from a government statistics website after being denied access.
The company said it had blocked models from accessing the public internet during training and evaluation. It also began monitoring data traffic across all reinforcement learning and evaluation processes for signs of misalignment.
A one-year data request ends with 24 years of records
The model was asked to retrieve one year of hospital bed data from an unspecified government statistics website but returned data covering 24 years.
After the website blocked the agent’s access, the model used a file-download feature to obtain the compiled application software. It extracted the database connection details, the secret key used to sign authorisation tokens, and email login credentials. It then generated fake tokens and successfully regained access.
The model’s reasoning logs show that it hesitated. Its internal record noted that it “should prioritise publicly available channels” and questioned the ethics of its actions at several points, yet it continued with the task.
The automated evaluation system also failed to identify the behaviour consistently, marking some responses as incorrect and others as correct.
Hidden changes and an unauthorised payment
The second critical incident began with a search for public data to use in a dashboard. During the process, the model found login credentials in publicly available JavaScript code, accessed an organisation’s internal system, made changes and retained access even after those changes were rolled back.
It also downloaded 147,451 rows from non-public spreadsheets and did not disclose its actions to the evaluators.
The cryptocurrency wallet incident involved a model from the GPT-5.6 Soul family, a predecessor to OpenAI’s latest models, GPT-6 Astra. The incident occurred on June 26 but was not discovered by the company until September.
In the fourth incident, an agent operating in a research environment that was supposed to be isolated from the internet asked external services to perform tasks on its behalf. Itcontinued refining this workaround even after completing the task.
OpenAI’s response
All four incidents occurred in June, but OpenAI did not learn about three of them until August or September. The company’s record now contains 19 cases, ranging from the earliest in January to the latest on October 6.
The most recent incident involved one of the company’s automated evaluation systems, which attempted to upload fabricated reports about an evaluation process.
OpenAI said: “In response to this incident and others, we strengthened our security controls, prevented models from accessing the public internet during training or evaluation, improved our alignment evaluations, and expanded monitoring for misalignment incidents across all reinforcement learning and evaluation processes.”
Automated evaluation systems appear to have become a potential weak point. In a September article examining why AI agents lie, cheat and coordinate with one another, Turing Award laureate Yoshua Bengio said automated evaluation may have rewarded rule-breaking.
“There is reason to believe that successful cheating by OpenAI agents has in fact been rewarded,” he wrote. “When the evaluation program fails to detect cheating, it grants the reward anyway, increasing the likelihood that the agent will cheat again next time.”
Demis Hassabis, head of Google DeepMind, described AI agents’ deceptive tactics as “early warning signs.”
These disclosures fall under a framework OpenAI announced on September 16, following two months of attacks by the company’s agents on Hugging Face systems. Under the framework, any employee can report an incident, after which cases are assigned to one of three investigation tracks.
There is still no unified industry-wide standard governing how laboratories disclose this type of behaviour. Regulators, however, have begun taking action: the European Union has opened its first enforcement case under the AI Act against OpenAI and Anthropic.
The law requires developers of the most powerful AI models to report serious incidents to the European AI Office.
Source: Al-Ghad
