Military aircraft had already taken off during the past spring, when US officials discovered something alarming: the intelligence underlying an armed operation against a Chinese ship had been fabricated by an AI chatbot.
اضافة اعلان
The operation was called off at the last minute, narrowly averting a potential confrontation with China, according to what CNN reported on Friday.
The incident highlights growing concern among military officials and outside experts, centered on the fact that decisionmakers' increasing reliance on AI could allow errors produced by these systems to climb up the chain of command before being questioned or verified.
The intelligence report, which circulated during the war with Iran, stated that the ship was carrying components intended for a nuclear weapons program.
The false intelligence originated from an analyst at Special Operations Command, who used an AI chatbot to gather and analyze open-source data alongside classified signals intelligence. But the chatbot misidentified the ship's cargo manifest.
The analyst then used the tool a second time to turn the erroneous findings into a summary that appeared official, before it was circulated through military command channels.
This incident comes at a time when the US military is rushing to integrate AI in order to speed up decision-making and maintain its edge over China.
The US Department of Defense has described AI as giving it a major advantage by accelerating what's known as the "kill chain," allowing commanders to respond in a timely manner.
But the very speed that makes AI attractive can also allow fabricated information, or "hallucinations," to slip through with insufficient human oversight.
Jake Stickler, a researcher at GovAI and a former US Army officer, said in a written response to TechCrunch: "It's important for members of the armed forces to understand the scale of uncertainty inherent in large language models. But this becomes even more important when it comes to decisions that could lead to the use of force, such as targeting, intelligence analysis, or operational planning. These are decisions that can carry life-and-death consequences."
Still, Stickler believes the incident should be a reason to add more safeguards to AI systems, not a reason to abandon them.
He added: "These tools can be useful in the right contexts and with the right safeguards in place. But prioritizing speed of adoption above everything else will likely lead to incidents that cause members of the armed forces to lose trust in these systems, which will ultimately slow down their adoption."
Resource: Al Ghad.
