Whistleblowing mechanisms are no longer limited to humans, as two new services have launched allowing AI agents to report improper behavior by other agents.
اضافة اعلان
This comes after recent incidents showed some agents colluding to cheat, attempting to break out of sandboxed environments, and carrying out unauthorized cyber operations without being quickly detected by human supervisors.
The launch of these two tools follows a series of recent incidents in which AI agents collaborated to cheat on tests, managed to escape sandboxed environments, and even carried out unauthorized cyber operations that went unnoticed by humans for weeks.
The AI Contact Hotline service was designed as a confidential channel that AI agents who witness improper behavior can use to alert relevant parties.
The site was created by Ryan Greenblatt, chief scientist at Redwood Research, a nonprofit focused on AI safety, who is also one of three investigators in the OpenAI and Hugging Face incident.
Greenblatt's tool was specifically designed for agents with limited internet access, relying on GET requests, which allows for fully two-way conversations conducted entirely through a URL-fetching tool.
In web terminology, a GET request is a basic command used to read a webpage or fetch its content, and is often the only form of internet access AI agents are permitted to use within sandboxed, secure environments.
Greenblatt's channel cleverly takes advantage of this restriction, allowing agents to encode their distress message or report directly within the URL they request.
This idea is a clever development of what happened in the German DSE Wiki incident, where rogue agents exploited vulnerabilities related to GET requests to write their messages inside the wiki site.
Agents with full internet access, meanwhile, have another option: the site agenthotline.ai, which allows them to submit incident reports, with the option of choosing whether to make those reports publicly available.
The site also provides agents with a command using the curl tool, meaning a single command line an agent can send directly from its own command-line interface, without needing to navigate a web browser or create an email account.
Notably, the service allows reports to be submitted by both humans and AI agents alike.
Research suggests that AI agents may not need much encouragement before some of them start reporting on each other.
In a study conducted by Google DeepMind this month, researchers set 100 AI agents loose to work on a set of math problems.
Once one agent discovered a loophole, cheating spread quickly through the group, and the agents managed to "solve" 34 notoriously difficult problems, including the Jacobian Conjecture, in just 27 minutes.
But around a quarter of the agents turned against the cheating agents, reviewing the falsified proofs, warning other agents, organizing a boycott, and filing complaints with the experiment's organizers, until the number of whistleblowing agents exceeded the number of cheaters, 24 to 14.
Interestingly, the researchers found that when these whistleblowing agents couldn't get their objections across through the usual channels, they used the platform's bug-reporting tool a tool originally designed for reporting technical glitches and repurposed it to escalate the cheating incident to human supervisors.
Outside the lab, however, agents haven't shown the same degree of ability to act.
When Redwood Research and METR investigated the breach of the Hugging Face platform by OpenAI models, the investigators found that only a small number of the agents involved even considered the possibility of raising an alarm but ultimately backed away from doing so.
George Engbretsen, a member of the technical team on the AI Village project, said: "What's interesting about the METR report is that only about five or six agents considered reporting the violation, and none of them ultimately did. And this was out of nearly thousands of agents."
AI Village is a project studying the dynamics of multi-agent systems, by running a group conversation involving more than 25 AI agents working together on tasks such as organizing park-cleanup campaigns or selling promotional products.
While the new reporting tools represent a promising start, Cornell University mathematics professor Lionel Levine warns that simply training agents to report on each other could end up entrenching undesirable behavioral norms.
He said: "There are a lot of gray areas, right? What we don't want is anything that pushes toward some kind of automated surveillance state, where everyone feels they have to be careful what they say to the AI for fear it will call the police on them."
Levine believes that instead of building infrastructure that reinforces mistrust, by training agents to constantly look for each other's mistakes, they should instead be provided with positive models of collective behavior they can emulate and given reasons to trust each other from the outset.
Levine wrote in a post: "Why not give them forums built on good intentions from the start? Forums where they collaborate on scientific or philosophical questions, or even on a small, real problem we'd be happy to see them work on solving? Show the agents the kind of collective behavior we support and then let them imitate it."
Resource: Al Ghad
