Former employees of Anthropic, OpenAI and Google DeepMind have warned New York officials about the risks of artificial intelligence and called for tighter oversight of its development. The warnings came during a hearing before the New York City Council, as local governments explore ways to address concerns that AI systems could pose a threat to humanity.
اضافة اعلان
Jacob Coxon, a former Anthropic engineer and an advocate for regulating the sector, said companies operate with a startup mentality: “Move fast, break things, and fix them later.” He added that this approach might work for a photo-sharing app, but not for developing “the most powerful technology ever built.”
According to The Associated Press, Coxon said AI could deliver major benefits, but a lack of transparency and safeguards increases the risk of things getting out of control. “On the current trajectory, I think the chance of humanity losing control of these systems is greater than not, and that could ultimately lead to human extinction,” he said.
Coxon was not widely known before resigning from Anthropic and publishing a series of posts expressing concerns about worst-case scenarios. Days after his resignation, prominent figures including Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman and xAI founder Elon Musk said they agreed on the need to slow AI development. The companies say they have taken steps to strengthen safety, but coordinated efforts to slow the sector face obstacles.
Disagreement between warnings and companies’ defenses
At the same hearing, representatives from OpenAI, Anthropic, Google and Meta testified that their systems help people in their daily lives and that safety is a priority for them. Anthropic and OpenAI also said advanced AI products need regulation, while company representatives expressed openness to some of the council’s proposals.
Morgan Dwyer, representing OpenAI, said: “I share your commitment to ensuring that this technology is developed safely and that its benefits are broadly shared.”
However, City Council President Julie Menin asked company representatives to estimate the likelihood of a catastrophe under the worst-case scenarios. Dwyer replied: “I don’t know. And I don’t think it matters whether the probability is 1% or 10% or 20%. None of those percentages are acceptable.
“We should not train models unless we can provide very strong evidence that we can keep them under human control.”
Menin described the response as “dismissive at best.” Dwyer responded that the percentage itself was not the most important issue, but rather the companies’ commitment to safety, which she said OpenAI is committed to.
Proposals to regulate AI
The New York City Council is considering several proposals, including requiring companies to disclose information about AI tools, requiring independent approval of systems used within the city, and giving whistleblowers a share of fines imposed on companies that violate regulations.
At the state level, New York Governor Kathy Hochul, a Democrat, is working on standards for AI reporting and transparency, in moves similar to those being pursued by other state governments. However, states’ ability to regulate the technology is partly limited because it extends beyond state and national borders.
Logan Graham of Anthropic said the company welcomes “smart regulation” and that local and state governments have a role to play in establishing appropriate frameworks.
“AI is powerful enough that mistakes in handling it will have real consequences for individuals, institutions and public trust. We must continue working tirelessly to develop scientific methods for reducing risks,” he said.
Resource: Al-Ghad.
