Our legal system isn’t ready for machines that act on their own.
49 comments
Technically, the code is owned and deployed by AI companies which make the AI companies 100% responsible. I'm not sure how is code written by LLMs different than any other kind of code. If somebody hacked just like OpenAI's agents did, he would've ended up in prison right away.
It's if nothing else a staple in the Arthur Conan Doyle tradition of mystery novels, and even then half the plots involved hiding who was culpable by using the machinations to create false alibis, with delayed murders and the illusion of a locked room as not even in the 19th century did anyone believe that the indirection itself would hold up as a defense.
If someone buries an anti-personnel mine in a public park and then argues that it acted on its own accord when it maimed a pedestrian, I doubt you'll find a court on the planet that would spend even a 4 seconds considering the culpability of the land mine itself.
You may have a case that the magic inference juice was somehow tainted, but that seems like a stretch. If you run your car into someone else, and it isn't a total failure of the systems of the car, you did that, not the car, and not the car company.
One, all major governments that have an AI presence in their borders have accepted that there is a high enough probability of a runaway self-improving AI advancement that will result in whoever owns it winning everything forever that they will do nothing to slow the development of the AI within their borders. In fact quite the contrary. Remember, they don't have to believe this is the only possible outcome, only that it is likely enough and catastrophic enough if someone else gets it first. And remember, they don't have to be right, either. It only has to be what they believe.
Secondly, the entire US economy is clearly tied up in AI. By extension, basically the entire world economy is too. The US is not the world economy anymore, but it's still a big enough fraction of it that if it goes down, everyone is going to go down. Every major government, in its own different way, needs the economy to stay up so the populace doesn't get antsy and so they continue to have money to spend. So, again, number must go up and if that means writing a blank check to the AI companies to break the law, so be it.
The good news is, barring the worst-case singularity outcome where the law doesn't matter anyhow, is that this won't go on forever. But I can't predict the exact time or way it'll cease being true.
That said, I'm very concerned about AI as such a tool. I used to work in RL and it seems crazy to me, the extent that the guardrails are dependent on RL working as planned. I don't personally believe that the tech is ready for what will be (and is) encountered in a dynamic unpredictable real world environment. What I've read from the HuggingFace 'transcripts' only amplifies my concern.
The company is obviously responsible for what their systems are doing
And of course all the ai companies are incentivized to do this. Their whole valuation depends on them being able to say their tools do this, can reduce labor and save money. If they aren't reducing labor, then that's terrible as these subscriptions are not insignificant amounts of money. It becomes less of a tool that can save money and more an actual new cost center that you really are just paying to appear to be keeping up with the joneses.
I wouldn't have trusted one of those agents to run without me watching the session log, let alone thousands.
We already have laws for this. If I misconfigured a pentesting tool and it breached an unauthorized target I'm liable. Why is this different?
Under what law specifically?
Just because you believe they should be held responsible doesn't mean the current law is written that way. CFAA charges require the defendant "acted knowingly or intentionally", you think openai intentionally acted to hack those sites? Would "they should have pretty much expected that output" stand up in court for criminal charges?
https://www.theregister.com/ai-and-ml/2026/09/14/ex-ftc-boss...
OpenAIs models since 5.5 were troublesome in ways even a layman like me could reproduce, their testing environments (“sandbox”) downright a showcase of what not to do and they, despite being one of the biggest labs, didn’t observe what any of their models output for weeks after multiple prior incidents. They had multiple warnings, they took not a single precaution.
You operate machinery or software, you are responsible to monitor it.
dns resolver fail?
Read the full thread on Hacker News →
Related stories
- Hacker News · 1 points · 10 days ago
- Hacker News · 3 points · 3 days ago
- Lobsters · 3 points · over 7 years ago
- Hacker News · 2 points · 6 days ago
- Music recommending system free and ready to usesimilaritiesengine.comHacker News · 2 points · 10 days ago
- Ars Technica · 0 points · 9 days ago