618 points•ibobev•2 days ago•275 comments•

275 comments

jimmyjazz142 days ago
> We must move past vague discussions of “AI” and instead isolate the specific types of systems that are creating problems.

This is absolutely correct and its surprising how rarely you see commenters in the media push to go into the specifics on this. AI is just matrix math and its what you connect that math to that matters, we should talk a little more about what we are willing to connect AI to and little less about how scary or capable the models appear.

yodsanklai2 days ago
A point that Yann LeCun has been making for a while is that it's meaningless to regulate AI, but we want to regulate applications. For instance, you don't want autonomous vehicles to hit people, whether they use AI or not. In practice, most applications of AI are already regulated.
truculent1 day ago
Why regulate gun ownership, if homicide is already illegal ?
thaumasiotes2 days ago
> For instance, you don't want autonomous vehicles to hit people, whether they use AI or not.

What would it mean for a vehicle to be "autonomous", but to "not use AI"?

perrygeoabout 10 hours ago
Exactly! What gets lost in these discussions is the fact that LLMs are models of language. That's it. They produce text which is completely inert, until someone decides to run that code, bake that recipe, or fire at that target.

Consider how trivial it would be to write a program that would destroy every computer on the world. Any competent developer could write this in a hour. An LLM in 5 minutes. The trick is actually getting the code onto every computer in the world and running it! Until someone grants it agency and runs it IRL, the artifact has zero impact.

The models can produce whatever scary text they want. But if a human acts on it, connects that with their email or their drone weapon, it's the human who bears total responsibility. We desperately need some legislation to enforce this; otherwise I see a future where almost any accountability can be avoided by AI-washing the problem.

Insanity2 days ago
“Commenters in the media” is fair - but Newport is also a compsci professor. He genuinely seems to understand what he’s talking about when you read his content or watch his videos.
throwitaway2222 days ago
But you can't stop someone from connecting AI to something at this point, so your point is moot.
egypturnash2 days ago
You could say it's illegal to connect AI to something, with heavy penalties if you're caught doing it anyway.

You could say that the human who hooked up an AI that did something that's already illegal is liable for the AI's actions. You could say that setting up an AI to do something illegal results in, ooh, an automatic multiplier of 10x on whatever the penalty for its illegal actions are.

Cthulhu_about 13 hours ago
You can't stop someone, no, but you can make it illegal so that at the very least, the major providers are compliant and anyone else will need to jump through the hoops. Like many "mildly" illegal things today like pirating films - it's easy enough if you know what you're doing, but finding and downloading a torrent file is still enough of an obstacle for >90%* of internet users

* number pulled out of my ass

eloisius2 days ago
Why does it need to come down to making it illegal to connect AI to something that influences real-world outcomes? Just do what we should already be doing: hold operators accountable for launching cyber attacks with a botnet.
Capricorn24812 days ago
I don't see anything materially different between me running a curl request that exploits something vs my local AI running a curl request. They're both programs I run on my computer. That's all Open AI is doing. They are hacking people from their computers. There's nothing complicated about it.
pclowes2 days ago
Yes you can, the US could shut down the entire internet if it wanted.

While that is extreme, if there is a high p(doom) it is also reasonable.

We can absolutely do things to control AI and fine or imprison those who can’t control theirs. This is not some inevitable outcome.

We could vaporize these companies tomorrow if we wanted.

iambateman2 days ago
Most of the commenters in the media are very much hoping the conversation doesn’t get to that level.
Animats2 days ago
This is a wrong-headed attempt to regulate AI. The real problems are different.

AI systems, especially multi-agent ones that can do things, are more akin to corporations, than individuals. When you read the logs from the Hugging Face incident, you're seeing something that looks a lot like internal corporate emails. Various units of the organization are arguing over what to do and who does what. They eventually converge and get the job done, breaking the rules at times. This is normal corporate behavior.

When agentic AIs do something outside their own internal world, they do it by engaging in transactions with external systems and people. As yet, few have robots to do their bidding. So, again, this is normal corporate behavior.

A corporation is a goal-seeking system. In theory, if you agree with Milton Friedman, its sole purpose is to maximize shareholder value. Internally, within the company, there are subgoals, which exist to support the top level goal. That, too, is what multi-agent AIs do.

The uncomfortable place this thinking leads is that AI regulation and corporate regulation are very similar. That's not something the political part of the world wants to think about too hard. It would mean putting more constraints on corporate power.

Yet that can't be ignored, because we're likely to see corporations where some parts are AI and some parts are human. That's already been done a few times as a demo, not too successfully. Yet. It's going to hit hard when an AI-run company outperforms a human one.

autoexec2 days ago
If we accept that AI systems are similar to corporations, then it seems clear that we must try to avoid the mistakes we've made in failing to hold corporations accountable for the harms they cause.

We should not only regulate AI more than we do corporations, but we should more strongly regulate corporations as well. There are corporations right now that are killing people, using slaves, and bribing governments just to increase the already massive amounts of wealth and power they have. If AI is showing any signs that it is going to act like corporations do we'd better act quickly.

jart1 day ago
It's not possible to regulate AI. Anticipating this would happen, I liberalized it a few years ago by turning it into a file that reads and writes text to stdio. Then I shared it with every man woman and child on earth. Now we all possess this great nuclear weapon. Good luck controlling that.
zdw2 days ago
"more constraints on corporate power" assumes that the existing ones are functional, which they arguably are not.

If we enforced existing laws against unauthorized access to someone else's computer resources against the companies who run a model that hacks someone else, rather than buying their "The agents did it, we're not responsible" BS, then maybe the incentives to behave responsibly would improve.

derekdahmer2 days ago
OpenAI immediately disclosed the hack and submitted to both internal and external investigations, and published extremely detailed breakdowns of what happened. How is that not taking responsibility?
crabmusket2 days ago
I agree with your framing of agentic AI systems as similar to corporations. That's interesting!

But what in TFA suggested to you a specific and wrong-headed regulatory approach? As I read it, the post calls to investigate the AI labs, to increase transparency of their operations. The linked NYT piece says:

> Before any intervention, Congress should lead a public fact-finding mission that reveals the unvarnished details of A.I. development.

Isn't this the first step to holding any corporation accountable, whether it is made of people or software?

Animats2 days ago
The paper calls for investigating the development process. Not constraining the deployment process. Use of AI systems by Flock and ICE, for example, is a deployment issue. Using AI to evaluate loan applications and resumes is a deployment issue.

The legal model to look at is the European Union's General Data Protection Regulation. That regulates what companies can do with data and how they can use it against people. You can develop anything you want in the data mining and analysis area, but the GPDR restricts how companies can use the results.

Focusing on the development process moves political attention away from what modest AI systems can do to people. They don't have to be super intelligent. Just super attentive. Always watching. Current technology is more than sufficient for oppression and control purposes.

This is the near term threat.

There's the threat to jobs, but that's something society has handled before. Some countries better than others.

Paul-E2 days ago
Corporations are made of humans. An airline may be a corporation, and it may take people's lives into it's hands, but when a plane crashes the pilots are the first one on the scene.
RandomLensman1 day ago
We have all sorts of regulations what corporations can or cannot do, who can run certain businesses, how certain businesses need to operate internally and externally - I don't think corporate power is the issue.

Also, not sure it makes a difference in relation to AI whether one thinks of AI more like corporate or a non-corporate thing to regulate. The basics might stay the same, e.g., who can do what under what procedures.

oblio1 day ago
> I don't think corporate power is the issue.

When were anti trust, anti cartel, anti monopoly laws last enforced?

lukewarm7072 days ago
This is an excellent article. I prefer to call them the AI giants or AI companies.

The companies and their employees must be held accountable: if the AI companies can't develop AI safely, they must cease their operations. If employees can't work safely, they must stop working.

Not enough attention is given to February 2026. That month, Anthropic changed its scaling policy roughly from A:

1 [To get the weapon I must endanger innocent others.]

2 [It is wrong to endanger innocent others.]

3 [I will not endanger innocent others.]

To roughly this instead:

1 [If I do not get the weapon, someone else will get the weapon.]

2 [I should have the weapon because I am the best.]

3 [To get the weapon I must endanger innocent others.]

4 [I should endanger innocent others because I am the best.]

This is based on the intuition:

[I should harm others to achieve my moral goals.] (act utilitarianism)

Anthropic has no mandate for these goals. I do not give them my consent to harm others on my behalf. I hope that politicians will communicate that these policies are not acceptable.

The employees of anthropic are responsible for what they do regardless of how much they get paid to do it. If they can't work safely, they must stop.

ozozozd2 days ago
It worries me that so few people realize that this is a known phenomena: https://en.wikipedia.org/wiki/Messiah_complex
tom2026hn2 days ago
“All of you may die, but that’s a sacrifice I’m willing to make.”
simoncion1 day ago
> I prefer to call them the AI giants or AI companies.

I prefer to call them "the major LLM manufacturers". They're companies like any other, manufacturing and selling complicated software like many others.

psyklic2 days ago
Why aren't they running agents on isolated computers without Internet access? This way, breaking free of containers would not matter.

It is truly a security nightmare that so many people are have given agents root access to their entire computers, in addition to presumably very private personal information.

specked-citrus2 days ago
Sooner or later, agents must be connected to the real world to do economically useful work. The sandbox failures are bad, sure, but the underlying misbehavior is the root problem here.
abeppu2 days ago
I think the way they're connected to the real world doesn't have to be as free-form as we're doing now.

If you think AI can help design new drugs, great! Let's pick some restricted DSL for describing molecules, preparation instructions, hook it up to the right set of lab robots that can pipette and centrifuge and whatever, _restrict its output to that DSL_, and let it iterate. The 'let an AI design drugs' process doesn't require that the agent can also, say, hack a niche German wiki.

For every area where we think these AIs can be _so valuable_ because they'll find solutions that humans wouldn't (or find them faster), then doesn't that value also imply it would be worth it for some humans to do some advanced setup of their specific domain-specific tools so it can be run in a sandbox equipped with only what it needs and not more?

NiloCK2 days ago
Because agents trained without internet access (real or simulated) would be bad at using the internet.

Specifically, bad at search and returning information with references, bad at discovering and debugging package versioning conflicts, etc.

It's a Metcalf thing. The utility of an agent grows in some fn of the tools it owns. Skilled tool use comes from rich training environments.

jackb40402 days ago
Anthropic claims that their intention with the agents that led to the Hugging Face hack was an offline experiment running against a hacking benchmark.

Either you're being pedantic and missing the entire point, or you're saying that Anthropic lied and made a fake sandbox knowing their agents would need to connect to the internet anyways.

malisper2 days ago
> Why aren't they running agents on isolated computers without Internet access?

They probably will soon, but even air-gapping may not be enough. See stuxnet for instance

DamnInteresting2 days ago
> See stuxnet for instance

I'll be impressed if AI agents find a way to get infected USB drives out into meatspace.

olejorgenb2 days ago
Likely because it's so much more convenient to allow (at least some) internet access.
tokioyoyo2 days ago
I think there’s a 2022 way of thinking how models are trained/evaluated, and there’s 2026 way of doing things. Models that are not evaluated with public internet access are pretty useless nowadays for daily operations.
mentalgear1 day ago
“We need to stop letting a small number of private companies, acting and talking in increasingly erratic ways, dictate how we’re supposed to feel about..."

The original quote follows with "AI", but it should be clear by now that you could put any topic here and the main issue is: private conglomerate money trying to control people.

And the remedies are well known: breaking corporate power and power concentrations through 1950s style regulation, as it was deployed against the robber barrons from back then and which resulted in a golden age for America and the world.

Read the full thread on Hacker News →

Related stories