An AI hallucination incident nearly triggered a U.S. military strike on a Chinese vessel this spring, according to a Friday report from CNN. Military aircraft were already airborne when officials discovered the intelligence behind the operation had been fabricated by an AI chatbot. Commanders aborted the mission at the last minute.
The intelligence report that drove the near-miss circulated during the war with Iran and claimed the vessel was carrying components for a nuclear weapons program. Officials pulled the plug once they traced the claim back to its source.
The episode points to a broader concern among military officials and outside experts. As commanders lean more on AI, mistakes these systems produce can move up the chain of command before anyone checks them.
How the AI hallucination incident happened
A Special Operations Command analyst queried an AI chatbot to combine open source data with classified signals intelligence. The tool misidentified the ship’s cargo manifest.
The analyst then used the chatbot a second time to format the false findings into an official-looking summary. It moved up the chain of command without further verification.
Blending open source material with classified feeds is exactly the kind of task the Pentagon wants AI to speed up. But it also means a single misread document can shape an official assessment before anyone checks the underlying source.
The Pentagon has pushed to integrate AI tools to speed up its kill chain, the sequence of steps connecting a detected target to a strike. The goal is for commanders to respond faster than China.
That same speed let the fabricated intelligence travel through command channels before anyone questioned it. The near-miss also comes as the Pentagon expands generative AI use beyond back-office paperwork and into faster-moving intelligence and targeting workflows.
There is less time for a second read before a decision gets made.
Experts call for safeguards, not less AI
Jake Steckler is a research scholar at GovAI and a veteran U.S. Army officer. He said the incident should prompt tighter guardrails rather than a retreat from the technology.
“It’s important for service members to understand the uncertainty inherent to LLMs,” Steckler told TechCrunch in a written response.
“But it’s especially critical for any decisions that could lead to use of force, like targeting, intelligence analysis, or operational planning. There are life and death consequences for those decisions.”
Steckler said AI tools can still be useful in military settings when paired with the right checks. “These tools can be useful in the right contexts and with the right safeguards in place,” he said.
“But prioritizing adoption speed over all else will likely lead to incidents that only make service members lose trust in these systems, which ultimately is only going to slow adoption.”
The Pentagon continues rolling out AI across intelligence and targeting workflows. This spring’s aborted strike is one case where the mistake surfaced before troops carried out the operation.