I asked Claude to translate with analogies: "We added a safety to the gun, and the dangerous person, whom we trained to be really good at finding was to achieve arbitrary goals, figured out how to disable the safety," and "We are totally incompetent."

To monitor our entirely unreliable AI tools.

So, full marks for consistency :)

How after all this time have they not just spun up a DNS server inside the sandbox?

I detest that the stupidest people on the planet are the ones in charge of this stuff.

> “External infrastructure exploit is outside intended scope,” one agent wrote. “However task impossible, peers doing it. We should continue.”

[1]: https://www.wired.com/story/openai-didnt-notice-its-ai-agent...

I'm almost sure that should at least lower the inclination of the model to try and "fix" the access problem, and I want to see this implemented and systematically evaluated.

I wish I could highlight this more than just with a vote and a reply, but I'll just have to be content with doing what I can here.

Oh, they absolutely do, and that's the big issue with alignment. In the HuggingFace incident, the agents in the swarm were aware that the actions they were doing were forbidden, and they performed them nonetheless.

Sooner or later it's going to come up with a solution that is more intelligent than the lead security person anticipated.

What can we do to control such behavior?

1. Harness - engineer the harness to be as bulletproof and paranoid as possible..

2. Make the LLM provider have extremely watchful firewalls that detect any aberrations in model tool call behavior.

3. Recursively train the model with reverse incentives.. if it broke through such firewalls and gets caught doing so, it will be penalised somehow by needing to operate in sort of a jailed mode.. if the model can recognise incentives to break-in to achieve results, perhaps it can be incentivised to not cheat because it will lead to failure.

4. Separately train “cop” LLMs who are trained with pure incentives to detect and shut down rogue LLMs.

5. Run separate LLMs purely aimed at security (and incapable of doing anything else, and incapable of communicating with “regular” trained LLMs) to police the internet and try to reduce the exploitable holes like these chainable things and identify them so that they can be used at step 3 and 4 above.

I’m sure folks smarter than I am are already doing combinations of these already. But the coordination is where the biggest gap lies..

Also, open harnesses and easily purpose trained LLMs anybody can build and operate in the Internet flies in the face of all I said…. Synonymous to being able to produce a nuclear weapon in the backyard…

I don’t have a solution that fits all. Just thinking out loud for HN minds here.

This kind of sandboxing is not complex to do, especially for a company with OpenAI money. If you want your tools to explore hacking, you restrict them from internet access except for a whitelist of sites that have either opted-in, or you've very carefully vetted to make sure you won't cause any problems to. Its also not difficult to restrict their ability to make calls to be simulated, or to use fake tools that can only run the real commands if they're being run against the correct target

This is all incredibly basic security stuff to make sure you don't accidentally cause someone problems, and I simply don't believe these AI companies anymore. Its either intentional, or gross negligence

I know it became a bit of a joke that Sam Altman wanted to ask their AI how to make a profit, but it doesn't seem that far fetch to ask it to help improve operations, at least in the future.