We already have a window into the future.

- Anthropic told everyone Mythos was dangerous because it's proficiency with biologics and cyber security

- Anthropic didn't release Mythos like everything else. They released a neutered fable. They didn't get rid of Mythos

- Anthropic opens a lab in SF

There was always a quesiton of "will the labs stop releasing their models and start building around them instead?" Yes - they already have. Anthropic is a biologic and cyber security company, in addition to intelligience.

Personally I wonder if they've been holding back a lot. Opus 5.5 was a good release after a little stagnation. Open AI releases good models and everyone says Anthropic sucks and -- Oh would you look at that - a better model finally and all of a sudden.

I always found it odd that people who are intelligent enough to work in IQ-loaded fields are as easily manipulated by either rhetoric or ideology as anyone else.

growing up I always assumed that everyone would grasp some aspect of game theory intuitively, I still remember the day I found out game theory was a thing - it was like finding out that someone successfully systematized common sense.

most people take the things people say as if they were worth considering. signals without cost are only useful as knowledge of what the signaler wants fools to believe.

> The motivations of people building the AI's are not the same as the people in charge of the labs. Looking at the last few years, has been a one-way door from OpenAI to Anthropic, and the main reason does not seem to be better compensation or even them winning, but mainly the fact they advertised to these employees that they would be the most careful when building this magic lamp. Their stance around 2023 / 2024 was one of the key reasons they were able to attract this talent.

> If recent news is to be believed, Anthropic culture even today seems to lean heavily on this effect

> In my opinion it has been the leading factor in getting and retaining the best employees who are often very worried about humanity dying to super intelligence.

I had not considered this viewpoint but this makes a lot of sense. A lot of Anthropic employees truly believe this and callout emphasis on safety as a key reason they work there. Now, Dario's post "Pacing the frontier" makes even sense -- it is as much for his employees than it is for the rest of the world.

This article is wholly flawed from the start where it claims nothing has been done, no actual effort made. A straight forward search of "what efforts were made and safeguards put in place subsequent to the CAISS statement in 2003?"

It also shows the same reasoning error mode many criticisms of a precautionary initiative or intervention to a problem:

Assuming that a problem whose trajectory was at a certain place when the initiative began has failed simply because it isn't solved on their own wished for timeline or standard of success, or that it wasn't meaningfully changed from what itnwod otherwise have been.

What happened to realizing there are hard problems, that different things may need to be tried, or that those things tried were partial but not complete solutions?

How about the simplest explanation?

* AI Labs hit the scaling wall. They need either new techniques, or vastly more powerful hardware to advance further.

This explains, the miraculous incompetence of AI labs in securing sandboxes and figuring out "alignment."

So they are between a rock and a hard place. They need limitless VC money because they cannot operate otherwise, and they do not have the capabilities to go further. The scare tactics and the "pacing the frontier" makes perfect sense then; they can IPO on the assumption that their ridiculous balance sheet doesn't matter because they are holding back. Because they are in control. The regulatory capture would be double whammy if they can manage it.

Open AI already said they have smarter models, and Opus 5.5 is rumored to be "taught" by a "teacher" model already; they are essentially distillations from bigger models, that both labs probably cannot economically serve to the public, due to hardware simply not being there. And, most of the improvements are not at the model level, but at the agentic glue level. Labs are getting better at RL'ing the models for agentic use cases, but the inherent flaws are still there. Models still have trouble with locality in writing for example (bunch of research on this that shows model size is the determinator), and agents are the bandaid over that.

And in the meantime if one of the labs makes a breakthrough, they'll push with all they have, because why wouldn't they? The idea that current LLMs can actually go rogue is just hilarious; in all cases, agents are being led by (deliberate) incompetence.

Pacing the frontier and the scare tactics will be seen as new generation's snakeoil tactics, perhaps will be called a flavor of AI CEOing or something.