Pentagon says overreliance on AI contributed to missile strike on Iran school

Pentagon says overreliance on AI contributed to missile strike on Iran school bloomberg.com

972 points•devonnull•8 days ago•552 comments•

#ai #pentagon #iran #missile #strike #overreliance #school #contributed

552 comments

> It found the U.S. “failed in its obligation to do everything feasible to verify” that the school was a military objective and that the failure “went beyond mere negligence.” The report said the United States “directed the strikes at the building of the school while being aware of a substantial risk of striking a civilian object and acting recklessly as regards the possibility that this would happen.”

Reading the details, "AI" doesn't really seem like the culprit -it's a scapegoat.

The intelligence that it was no longer a military target never entered the target database, the team that was responsible for vetting the target list was gutted, and said team was never even consulted.

The White House wanted 1000 targets and pulled from their database without any due diligence. Whether it was an AI call or an SQL query - this was from pure human maliciousness and incompetence.

oooyay8 days ago

I'm not sure if it was any kind of official rule, but I worked at times in a JTAC capacity for troops in contact. I mention the last bit because it's very high stakes where seconds and minutes count a great deal more than when you do coordinated tactical strikes against strategic objectives like weapons caches. When I did that kind of work we had to have three eyes-on forms of contact. Often that was the calling troops, the observer (like an Air officer), and an air asset like a drone. They all had to independently describe the target, its orientation, and surrounding activity.

However you sourced your target is irrelevant to the activity at hand. When I first heard of this news I knew immediately someone skipped target verification or that it was simply no longer a policy of the DOD.

somenameforme8 days ago

Famously, the last 'enemy' we killed in Afghanistan, alongside the 7 children and others surrounding him, was somebody who was working for a US aid organization. [1] A drone operator saw him putting bottles of water into his trunk, and somehow thought they were bombs, so they tracked him for 8 hours, then killed him and his family.

[1] - https://www.bbc.com/news/world-us-canada-58604655

cicko8 days ago

Yes, because it's DOW now.

0xWTF8 days ago

> I worked at times in a JTAC capacity

glad to see you're alive and functioning

puelocesar8 days ago

I always said that AI is dangerous not because it's superintelligence will threaten humanity, but because some dumb/lazy people will overestimate it's abilities and use it where they should not.

This is just one more example of it. I'm waiting for even more disastrous ones.

goodcanadian7 days ago

That has long been my fear as well. I regularly encounter people who have given up all pretense of critical thought in favour of trusting some buggy computer system. And this started long before "AI" becoming prevalent.

jackb40407 days ago

All the AI leaders running around demanding everyone care about hypothetical "existential risk" and killing all of humanity are suspiciously silent on its active role in the emerging world war.

xlayn8 days ago

Now, ask yourself if you want AI taking decisions around your life... it will pull numbers... someone will have a "goal" of 1000 "hits" and there you go, 2-3 months in jail working to check what went wrong...

But don't worry, the other P AI system will ensure that the first one doesn't do mistakes...

toomuchtodo8 days ago

Lawsuit Says UnitedHealth Used AI With a 90% Error Rate to Deny Care - https://www.yahoo.com/news/us/articles/lawsuit-says-unitedhe... - September 18th, 2026

UnitedHealth uses AI model with 90% error rate to deny care, lawsuit alleges - https://news.ycombinator.com/item?id=38299921 - November 2023 (58 comments)

https://litigationtracker.law.georgetown.edu/wp-content/uplo...

TheOtherHobbes8 days ago

You seem to have missed the point, which is that the decision to murder a large number of innocent people was made by humans at the top of the chain of command, and 'AI' is being used as a thin pretext for that.

The pattern matches various other acts of senseless violence against innocents in South America.

The message isn't "We did this accidentally. We should understand how it happened, so we can make sure it doesn't happen again.".

It's "We do this to make the point that we can, no one can stop us, and if we're challenged we're going to lie about it."

8note8 days ago

what makes ai special here? a person following an SOP that involves polling an incomplete dataset will have the same 2-3 months in jail

throw0101c7 days ago

> Now, ask yourself if you want AI taking decisions around your life... it will pull numbers... someone will have a "goal" of 1000 "hits" and there you go

Before anyone does this please let me know beforehand so I can invest in paperclip factories…

seizethecheese8 days ago

Important context is that the notes about the changing use of the location weren’t even in a connected system. Unclear if the human analysts would have even caught this, or if they would have known to check this unconnected system.

> Some US intelligence analysts took notice, the officials said. Bloomberg reported in June that one analyst detected changes to the site as early as 2019, and logged remarks about the modifications in a system that wasn’t connected to the primary military intelligence database, which informs targeting

pbhjpbhj8 days ago

>"Unclear if the human analysts would have even caught this" (seizethecheese)

At the time a news outlet put out commercially purchased images of the target site before and after -- even a layman could see that it had a children's playground next to the building, and that it was separated from the compound.

https://www.channel4.com/news/factcheck/factcheck-iran-schoo... has the info from around the time, but doesn't show the detailed images in the news report (video) I seem to remember seeing.

I think it was maybe some earlier coverage from NYT, they published this retrospectively - https://www.nytimes.com/video/world/middleeast/1000000107503... (which is also on YouTube, https://www.youtube.com/watch?v=hDcim48W4Io). It shows that even a cursory analysis - without the benefit of enhanced military grade satellite/flyover imagery - indicates the building targeted with pin-point accuracy is a school (eg https://youtu.be/hDcim48W4Io?t=98).

In short, this seems entirely avoidable. I don't think you need any connected system, you just need an image of the target and eyes.

If they trained a model to find targets and didn't train it to avoid civilian targets, and couldn't be bothered to have anyone check them, that's still a war crime. The analyst could probably see the kids playing there with the imaging tech they have available.

This war was unlawfully started by the military at Trump's command, knowing it was not a lawful order; just as they knew firing on shipwrecked civilian sailors was murder. Seemingly it was started to win money betting on fuel prices.

jauntywundrkind8 days ago

"A computer can never be held accountable / Therefore a computer must must never make a management decision" https://simonwillison.net/2025/Feb/3/a-computer-can-never-be...

Except laundering our biases & discriminations, the unaccountability seems to be the purpose of a huge huge swarth of AI usage. To eliminate accountability. That sure seems like A Feature Not A Bug to the worlds worst people.

kamranjon7 days ago

Many people are commenting that AI is just a scapegoat here and really this is a story about military incompetence. What I think is probably more interesting is how AI enables incompetent people to do more damage than they would otherwise. Two things can be true at once, the military can be incompetent, and AI can be used to make targeting decisions that are wrong - we can understand both the new world that we live in, and the absolution of responsibility it offers without ignoring these nuances.

int0x297 days ago

AI enables more mistakes to be made faster and more convincingly. You have more stuff being generated which means more of it (by absolute quantity) is probably wrong. You have less time to review it because there is more material. Of course everything the wrong stuff included can have analysis making an argument for it attached (courtesy of an LLM). And finally people writ large tend to treat computer output as infalliable.

This increases not just the risk of war crimes and general moral bankruptcy but also of escalation (see the recent Chinese ship close call or the older, and not AI related, bombing of the Chinese embassy in Belgrade due to faulty intelligence) Also dropping multi million dollar missiles on non legitimate targets is a giant waste of money and more human analysis is likely cheaper.

jackb40407 days ago

The concern in this story is whether the targeting was actually a mistake, or if we intentionally killed a school full of children. If we take AI out of the equation, and it's just a soldier who made the call claiming it was an accident, we might not just believe that on its face. But throw AI in the mix and suddenly the media loses all ability to question anything they read in a press release.

HlessClaudesman7 days ago

Guns don't shoot people, people shoot people.

Perhaps AI is a gun.

feoren7 days ago

The danger of AI has always been the recklessness and maliciousness of the humans who wield it, rather than some Skynet or paperclip maximization fantasies. So yes, just like with guns, with mass media, with any other powerful tool.

iAMkenough7 days ago

A great big swarm of AI guns in the sky?

k3108 days ago

Opinion: AI was set up as the fall guy from day one.

The murders continue and a certain mindset demands a fall guy for everything.

Iran itself was blamed.

This was a criminal activity ultimately in the hands of humans, explaining the extreme sanctions being placed on the ICC.

darkoob128 days ago

To my view Iran government is responsible like 50% for turning part of a military compund into a school. That was irresponsible and shows how little they care about the life of their citizens.

culi8 days ago

The US has more public schools on military bases than any other country. Over 160. And that's not counting the private schools

k3108 days ago

Speaking of human shields, if that is the case, and not swords into plowshares, how about using Air Force 1 as a potential sacrificial target for the skulk jet?

There is no honor among warring nations.

emsign8 days ago

50%? Not 51%? Or 49%?

impossiblefork8 days ago

Personally I'm more interested in the justification for the second and third strikes.

There needs to be some protection for first responders and others helping the injured.

FireBeyond8 days ago

Israel likes to argue that Hamas embeds fighters with first responders, which even if so, they're still responding to injured. But Israel considers this not even collateral damage but justified.

Mind you Israel is currently funding a lobbyist organization in the UK to flag Médecins Sans Frontières (Doctors Without Borders) as a terrorist group (because many years ago a couple of people who had affiliations with other anti-Israel groups were found to be also affiliated with MSF).

jackb40407 days ago

Even if that weren't a cruel joke, what would be the tactical purpose of soldiers embedded with first responders rushing to the site of an airstrike? They go stand on top of the dead bodies and fire a rocket launcher at a departing fighter jet?

IDK, maybe it's pointless to argue at this point in history. You can probably count on 1 hand the number of people on the fence. The majority of humanity is on the right side, they just have no ability to influence their governments. Arguments like this aren't supposed to convince anyone, they're just something to be repeated over and over to prevent critical thinking from taking place.

impossiblefork8 days ago

Yeah, I've heard about it, but presumable Americans soldiers still care about IHL at least to some degree, even though they've historically not bothered with the Geneva convention and have tortured people.

FireBeyond8 days ago

Responding here to answer the dead comment which feels entirely disingenuous.

No-one mentioned "using ambulances as cover". That is separate from "enemy combatants are acting in a first responder capacity, but are still enemy combatants, therefore the entire first response effort is a viable target". That is a war crime that Israel regularly likes to commit.

> These are the same people who view women as property, homosexuality as a reason for execution, and 13 years olds as viable brides.

Halacha? Acquiring your wife through kiddushin? Only a husband can issue a get (divorce document)? Women in agunahs (dead marriages)? Sounds like property to me.

Rabbinic law makes capital punishment effectively impossible, but the Orthodox sects do everything possible to punish: disownment, both family and social and actual exile from physical communities.

Don't girls reach halachic adulthood at 12?

And lets not throw stones here. Even in the US, you can get married with parental consent or a judge's consent at any age in Mississippi, New Mexico and Oklahoma.

You can get married at 15 in Hawaii and Kansas.

You are "grandfathered" into sexual consent (not as in "accepting of marital rape" but "it's no longer statutory rape if she's your wife") when you do so.

So we may not want to be lording it too much on some guise of moral superiority, particularly given that most of the marriages that happen at those ages are not "young couple got pregnant, want to do the right thing" but "older man in conservative/religious community saw young vulnerable girl and decided he had to have her".

drakonka8 days ago

In this case it sounds like AI may be more of a scapegoat. The incident of US almost boarding a Chinese ship due to an AI-assisted intelligent report claiming it was transporting nuclear weapons components[0] was more compelling for me in terms of direct AI usage impact. Though the primary issue seems more one of people carelessly offloading this type of work to AI in the first place (or being pressured to do so?)

[0] https://edition.cnn.com/2026/09/18/politics/us-military-ai-f...

neuronexmachina7 days ago

> The incident of US almost boarding a Chinese ship due to an AI-assisted intelligent report claiming it was transporting nuclear weapons components[0] was more compelling for me in terms of direct AI usage impact.

To be fair, there was a rather similar incident in the 1990s, so that sort of thing can happen without any AI at all:

https://en.wikipedia.org/wiki/Yinhe_incident

impossiblefork8 days ago

Surely that is just the US asserting themselves to be more belligerent than they are? Just a bluff?

Imagine if China seized a US ship carrying nuclear weapons components. It'd be war immediately, and the US of course knows that the same is true of China.

There's also no basis under international law. China is of course a nuclear weapons state and numerous nuclear weapons states have transported nuclear weapons around the world, with nuclear weapons sharing being a something old and accepted.

One special reason that one would expect China to respond to such an operation by actually going to war immediately is that, if it were a nuclear weapon, with all the details and delivery systems etc., it could be analyzed to see how it reacts to missile defence stuff, which might reveal some unknown weakness that would allow MAD to be ended. So somebody getting themselves access to the core reentry, MIRV and maneuvering stuff would be a reason to attack him right away.

Read the full thread on Hacker News →

552 comments

Related stories