You use a bunch of technical-sounding words here to make it sound like you understand. But to be clear, nobody understands why the evolved weights of a NN make the decisions that they do.

Almost nothing is understood about the actual representations used for nontrivial concepts, decision algorithms, etc.

If you look at the field of mechanistic interpretability, compared to “GOFAI” like learned decision trees, an LLM is completely opaque.

> To name it "hallucination" is an euphemism... those are errors

I find this and other "don't anthropomorphize the computer" statements incredibly unconvincing.

People develop terms for things and language has always contained overloaded or "literally inaccurate" terms.

An LLM can have "hallucinations" in the same way a modern computer program can have "bugs".

Lane Kiffin almost destroyed LSU's football program acting on legal advice from ChatGPT. A video game publisher owes the former owners of a studio it acquired $200+ million because he based his actions on legal advice from ChatGPT. In the past week alone, California has disciplined over a dozen attorneys for LLM hallucinations because they used LLMs (mostly ChatGPT) to produce their legal pleadings.

And that's in an area where there are multiple safeguards to catch the issues before they become permanent problems. There's absolutely no justification for using AI in warfare, where mistakes tend to be pretty final.

Then there's old-fashioned F'ups that don't fit your political agenda and are often quite damaging and embarrassing, not to mention lethal for people who don't deserve it. e.g. The U.S. used AI tools meant for rapidly picking targets in the middle of a war to plan their initial strikes on Iran. They had time to double check everything and do their due diligence before striking, but they didn't. So, a school next to a military base was targeted and a lot of kids died. This was a genuine F'up resulting from relying on a tool meant to give rapid but merely okay target selection under time pressure when there was no time pressure. The real mistake was made by humans.

The current case of the mistaken nuclear weapon parts shipment seems like an old-fashioned F'up, updated for the times. The people who didn't simply trust the tools and actually double checked should be commended. Others in their situation wouldn't have. I fully expect AI will be scapegoated for a lot of similar F'ups in the future even though it's still the responsibility of human beings to use ethics, caution, and restraint. AI doesn't get fired. Doesn't sue. It's actually pretty awesome for taking the blame.

Or the War Games movie and the Norad training mistake that inspired it.

The PRC has been hard against nuclear proliferation as a policy over decades, it is highly compliant with IAEA inspection norms, despite the NPT not making it mandatory to be under those inspections. This policy is not something the US has in the past or will in the future engender into it through force.