Donald Trump signs voluntary AI safety pact as California’s Gavin Newsom rebels

Donald Trump has signed a voluntary AI safety agreement with the heads of the country’s leading technology companies, setting up an immediate clash with California Governor Gavin Newsom over how to regulate the rapidly growing sector.

The White House accord leaves the responsibility for policing artificial intelligence almost entirely to the tech firms themselves. Speaking in the Oval Office, Trump made it clear he has no intention of imposing binding federal rules. "They’re gonna police, and they’re gonna police each other, and they’re gonna self-police," Trump said. He has also championed a rhetorical rebrand, pushing to replace "artificial intelligence" with the term "superintelligence", and suggested the FBI and the Department of Justice are guardrails enough for the industry.

This hands-off approach met immediate resistance from California. Governor Newsom signed an executive order mandating that all state agencies continue using the term "artificial intelligence", pointedly dismissing the federal push. Newsom’s order called the name change a distraction from a "craven failure to take action" on documented security risks. "We have to do a lot more in the absence of federal leadership," Newsom told reporters, leaving open the possibility of calling a special legislative session to address the issue.

Silicon Valley split over risks

The political divide is mirrored by growing friction within Silicon Valley itself. While some executives welcome the lack of regulation, others are raising alarms.

Anthropic chief executive Dario Amodei has faced private pushback from other tech leaders, including Nvidia’s Jensen Huang, who confronted him over his public warnings about the existential risks of AI. Trump himself weighed in during a recent media interaction, telling Amodei to "be careful" with his warnings.

But critics argue that the actual dangers of AI are already here, and they look less like sci-fi superintelligence and more like basic software errors in high-stakes situations.

Chatbots on the battlefield

A CNN report in September revealed that the US military came perilously close to boarding a Chinese vessel—an action that could have triggered a major diplomatic or military crisis—after relying on an intelligence report generated by an error-prone chatbot. The chatbot had falsely claimed the ship was carrying nuclear weapons components.

Despite these risks, the US military's reliance on these systems is growing. Trump reportedly spent hours consulting Elon Musk’s Grok chatbot during a secret meeting in December 2025, asking how the public would react to the capture of Venezuelan President Nicolás Maduro. The US invaded Venezuela on 3 January, and the Pentagon subsequently used a specialised version, Gov Grok, during the war in Iran.

Musk and Anduril’s Palmer Luckey have since been appointed by the Pentagon to lead a study on how advanced technology can be used on the battlefield.

With the federal government stepping aside, the task of setting hard limits on how these tools are used is falling to state capitols and internal industry battles. For now, the Trump administration’s plan is to let the companies write their own rules.