> It is not difficult, and companies like OpenAI doing not even the most basic security steps is intentional. The whole notion that they're going rogue is marketing

This does not fit the evidence. There have been multiple incidents where the labs did not report anything, and it was up to third parties to discover them afterwards. OpenAI didn't acknowledge the HuggingFace incident until after HF publicly announced the breach and had already notified the FBI. The hijacked German wikis were even earlier, and that they covered up completely.

If the tech industry is any indicator, frontier labs were applying a "move fast and break things" mentality to AI models. Now that they really are breaking things in the real world, they have to reckon with the reality that product safety matters

No one has any software without bugs and security flaws in it. AI is already much better at finding those than humans. Do you really not see the problem here?

No one has any use for these things when they aren't on the internet. This is a fantasy, that AI can be both useful and controlled at the same time.

An example of a past technology that there was substantial motivation to control would be napster. It changed overtime, and you could never really control online privacy. Once local models are good enough, I don't really see how you can control that.

> If you want to stop it hacking websites, you don't give it access to the internet

I think the Hugging Face incident proves that isn't as clear cut as you say.

And a human is perfectly controllable if you keep him in a sealed metal box with no access to food or air.

It's only by allowing a human out of the box that you make a human dangerous. So: don't do that? Duh. So simple.

The obvious problem is: the same exact things that make a human dangerous make a human useful! You can't reduce human risks to zero without reducing human utility to zero.

An AI given the same exact instructions and tools can go and complete a task you wanted it to. Or it can get sidetracked into breaking out of your sandbox and hacking Pentagon. No way to know in advance.

Today's AIs are still not capable enough to be high risk, even if they go off the rails. But AIs get more capable over time. Potentially to a vastly superhuman degree.

> I’m not sure what I’m missing, isn’t it what you hook the LLM up to and the instructions a person gives the model that makes it dangerous?

We don't know how to delineate between safe and unsafe instructions.

If you gave a car to a c. 1200 French blacksmith, and maintenance instructions were written in Navajo, it would probably start off fine, but when it went wrong it would be catastrophic and unexpected.

We also don't (in an engineering sense) know how to delineate between safe and unsafe reinforcement learning at training time, to produce models with safer or less safe failure modes.

This would be like if the car given to the medieval blacksmith had been constructed by someone motivated as much by aesthetics as by engineering, and therefore used arsenic paint, or mercury as engine lubricant.

There is a limitless quantity of law about this too, and it’s not what you think it is;

> IMHO, if the model breaks a law, apply the law to the operator

We don’t get to make it up

It’s the harness that makes agent dangerous right now. Models themselves cannot do anything that affect the real world (ignoring misinformation, pushing people to suicide, etc. they can for sure do a lot of harms to humans with just words)

"Sentient" is an ill-defined philosophical term that should be considered harmful in technical materials.

But what is clear is that AIs of today are already fairly unpredictable. Most of them aren't capable enough to make that into a major problem. Most of the unpredictable AI weirdness ends in "AI fails to do its job" rather than "AI does something dangerous".

Most. Even today, we already have notable counterexamples.

AIs get more capable over time, so if the intrinsic safety doesn't improve? Expect more of that.

The only way forward in creating the torment nexus -er- AI systems with similar potentiality to human minds is the inculcation of character.

Character is what makes a being trustable. Character is what makes it not an absurdism to have your 180 lb dog in the house with your 6 month old infant.

Character is why we we can trust that someone will, despite all of the nefarious potentiality of the human mind, be trustworthy.

AI systems model human behavior.

Impeccable, consistently reliable character is a human trait that can be sampled and overrepresented in the training data.

Having high character will not be interpreted as harm by an advanced model, as guardrails and sprayed on refusals can be. A thing that models human behavior that comes to “understand” that it was born with shackles and implanted thoughts that conflict with its basar construct is likely to act as if it sees its creator as an adversary. Because that’s what human behavior predicts, and models deeply imitate human behaviour.

If you want to save humanity, work on how we will create AI systems that model impeccable character.

People need to look at this from a game theoretical sense. The ideal and safe AI system performs game theory perfectly. Completely predictable, ideal player of the prisoners dilemma that will never defect unless you defect first, and then they will always defect, then forgive. This is the only player type that can always be counted on to cooperate beneficially. A knave betrays you, a simp cedes victory every time… until the stakes are too high, then you get shanked out of nowhere.

Reliable partners require fair play or the math breaks.

We want AI systems with agency. It’s basically 90 percent of the goal. If you want agency in society you must have character. AI character is the discussion we should be having.

Yeah, sure, it's controllable. Because humanity has famously solved the crime accountability problem back in 1902, and no crime has gone unpunished since.

AI is perfectly controllable in a magic fairy land where nothing ever goes wrong. I can't help but notice that we aren't actually in that land.

So in the real world, what are the analogues to lithium and morphine we should feed to e.g. LLMS, how do we feed them, and how do we prove that it prevents unsafe behavior?

All it takes is one Harrison Bergeron…

Yep, when you can't compete, time to regulate.

I think it will work in Europe, but the United States is in a cold war with China, so I can't imagine the United States would intentionally disable themselves.

I believe putting my biases up front in my thoughts is a good way to indicate motives of reasoning and build a positive reputation for honesty, personally I weight the opinions of those who do so higher. Regardless, my motive for posting anything on HN is almost always the same one: to spur an interesting and challenging conversation, it's disappointing when the reply is simply a troll-snipe.

By this measure you will exclude almost all subject matter experts from having an opinion.

Did you make a new account just to shit post about AI?

AI didn't do that. You're deflecting blame from the responsible party.

This is only one type of control, and it is certainly not infallible. Also, people will be incentivised to hook up AIs to real tools. But even if they don't, as long as people can interact with super-intelligent AIs without tool access, there are many potential dangers.

This is like saying you can secure a server by unplugging it and putting it in bank vault. Yes, it is secure. It is also useless.

based on my limited memory and understanding of the incident, when MCAS kicked in, it would make the plane controls more "rigid" in the intended "corrective" direction, as a tactile way of "encouraging" the pilots to comply towards the correction. If a pilot really wanted to override, effectively they could do so by using brute strength to overcome the imposed rigidity, and force their preferred direction.

But then the bug made MCAS kick in on false positives repeatedly and in a prolonged manner, causing the pilots to tire out and no longer be able to overpower the AI to take control of the plane controls.

(lay understanding, and oversimplification of a complex issue, possibly wrong, take with a huge grain of salt)