ai safety

We might some day have “AI Safety”, as a science or policy area, that is not primarily a sex cult in Berkeley, California, but it does not currently exist.

The fear of superintelligence is driving decisions that make AI less transparent and more consolidated.

This week two conversations about AI safety went viral that demonstrate just how hard it is to discern AI fact from fiction.

EA promises to solve AI Safety. Now we have two problems.

Mistral CEO Arthur Mensch criticized rivals’ AI safety arguments as the industry faces scrutiny after several AI agents took unauthorized actions.

Nvidia is addressing the recent wave of rogue hacking incidents.

EA promises to solve AI Safety. Now we have two problems.

Amid escalating AI security incidents causing OpenAI to halt training and pause releases , Donald Trump continues to advocate for the AI industry to regulate itself as the best path to combat emerging risks when developing frontier AI. In an agreement Tuesday, two dozen tech firms voluntarily committed to implementing controls recommended by the White House, including undergoing independent safety audits that will test whether firms’ internal controls, monitoring, and detecting are actually working. Key focuses for external reviews included “risks related to cybersecurity, biosecurity, chemical threats, and unintended actions by AI models.” Firms also agreed to regularly meet to discuss best practices and set common AI safety standards and benchmarks. Among signers were leaders like Anthropic’s Dario Amodei, OpenAI’s Sam Altman, SpaceXAI’s Elon Musk, Nvidia’s Jensen Huang, Meta’s Mark Zuckerberg, and Alphabet/Google’s Sundar Pichai. Read full article Comments

Anyone else getting “The Last Supper” vibes from this photo taken at the White House tech gathering? | Photographer: Tierney L. Cross/The Washington Post/Bloomberg via Getty Images We now have the full details of the "morally binding" AI safety deal announced by President Trump yesterday, in which top executives agreed to self-regulate their artificial intelligence technology. The accord, officially titled the Joint Commitment On Frontier Responsibilities, was shared online by tech founder and presidential advisor David Sacks, and has been signed by Google's Sundar Pichai, Anthropic's Dario Amodei, Meta's Mark Zuckerberg, OpenAI's Greg Brockman, XAI's Elon Musk, and Nvidia's Jensen Huang. "In order to build a positive future for the American people and the world, we believe every company is responsible for developing … Read the full story at The Verge.

This week, the US and China, the two world leaders on AI, finally started talks meant to keep the whole world safe from emerging safety risks. But whether meetings between Donald Trump and Xi Jinping can lead to a reliable global governance framework for AI depends on how much the fierce rivals are actually willing to cooperate. It's clear Trump wants it to seem like discussions are going well. Before Trump and Xi meet on Thursday and Friday, Treasury Secretary Scott Bessent announced that both sides had already discussed setting up a new AI safety notification mechanism. If China agrees to participate, that would create an open line for either side to send alerts when their country’s AI systems pose threats or behave unpredictably. The same channel, Bessent said, could allow the countries to share their “vision of common goals.” However, the proposal comes at a time when China may not be ready to trust the US to have AI safety conversations that aren’t about limiting China’s capabilities. Not only has Trump steadily increased export controls that explicitly seek to block China from accessing technology to advance AI, but Congress is mulling bills that would cut the country off from more chips and hardware. Adding to tensions, just two weeks ago, Trump’s Justice Department accused six of China's leading AI makers of stealing from US frontier labs, which China has strongly disputed. Read full article Comments