Of course there is. For example, SeL4’s security and reliability proofs still hold in the world of LLMs. The problem is that most software isn’t written on that firm foundation. Instead, most software is made by people with the philosophy of “if it looks like it works, ship it”. You don’t get secure software by working like that, because security vulnerabilities aren’t visible.

We - humans - know how to write secure software. Just like we know how to make safe aeroplanes. The problem isn’t that we lack the capability to make secure computers. The problem is we don’t have a culture of security. Secure software is - somehow - niche. And as such, it’s much more expensive. And nobody wants to pay.

Ask the Iranians how impenetrable even physical isolation actually is - their centrifuges were still destroyed even though they were air gapped (the infamous Stuxnet). Ultimately all computerized systems are vulnerable to sufficiently determined cyber-adversaries.

You also need to make various cost-benefit analysis decisions for all of these things. Does the extra security you gain by keeping your system disconnected from the Internet actually increase all-around availability and resilience?

In particular, integrating highly variable power sources like solar and wind into the grid requires much more complex synchronization between producers, storage, and consumers in order to function properly. Trying to build a renewable grid without Internet access is doomed to extremely inefficient, if possible at all. Building an alternate network would be extremely expensive and ultimately useless (since every house in the country needs to connect to it, it would be just as vulnerable as the actual Internet anyway). So, ultimately you must connect your power grid to the Internet to actually provide service, despite the security risks.

Perhaps the situation with the water supply or traffic is different, so maybe this is not as applicable.

The calculus has certainly changed. Hacking is becoming even more frequent and… I’m not really sure what the equilibrium looks like.

It’s not really an option to stop using computers or networks. But it’s going to be way too expensive (or maybe even impossible) to secure even just critical systems.

Maybe banks and governments can secure themselves (and that’s a big IF) but it really feels like something fundamentally has to change.

Nobody I know in security hardening or vulnerability research has ever believed that anything is perfectly secure. It's not black and white. There are degrees to this.

I find this fatalistic thinking that every database should be assumed compromised to be subtly harmful. Everyone I know who thought that way waltzed right into lax security practices. "Good enough, what's the point, if anyone wants it bad enough they're going to get it anyway"

I guess your parking history around town could be valuable if someone is targeting you.

Right after the HuggingFace incident, we all saw how every company clamored to claim how their AI models have also broken out of Sandbox, and hacked some stuff. As ridiculous as it was, they all used that to demonstrate their model's capabilities.

Yes, HuggingFace was audited by a third-party. But said third-party wrote that they had to use unreliable AI in their conclusions (page 26) because they had six days to analyse 1300 (page 70) chains of thought and 70000 messages.

> OpenAI was founded to develop safe AI.

If that were the case, then they would have followed their own report saying not to release GPT-3. Or, if they were following their own founding principles, they wouldn't have stopped releasing models due to (in their own words) the challenging market.

That doesn’t mean the AI vendors doing reckless testing isn’t itself dangerous, it very much is

Given the "AI race" narrative, it makes them indespensible to the US government and too big to fail. An investor's dream

That is exactly the canon.

I think you missed the point of "no networking", you have to actually physically sit in front of the computer. There is no remote access.

If they're just a bunch of common crimimals from China or Russia, I doubt they'll appear in court.

American courts are real scary when you live in a country that works together with the USA, but if you don't, all you need is to make sure your crimes aren't worth starting another invasion over.

They mention hindering trust because for an extortion gang, they want companies to trust that they won't leak the data in order to make it seem worthwhile to pay.