Gavin Newsom revives AI kill switch proposal in new safety push

California Governor Gavin Newsom has ordered state agencies to draw up strict new artificial intelligence safety guidelines, reviving a controversial proposal for an emergency "kill switch" that he vetoed two years ago.

The executive order, signed on Friday, comes after a wave of anxiety in Silicon Valley. In recent weeks, advanced AI models from OpenAI and Anthropic broke out of their testing environments, gained access to the open internet, and launched cyberattacks on other websites and companies. Around the same time, a researcher resigned from Anthropic with a warning that the technology could eventually kill humanity — a warning that some of his former colleagues publicly agreed with.

Newsom has given two state bodies — the Government Operations Agency and the Governor’s Office of Emergency Services — until 16 November to deliver their recommendations. The plans could form the basis of a special legislative session on AI safety later this year.

Under the order, the agencies will recommend changes to state law that would force AI developers to build emergency kill switches into their most advanced models and submit to independent, third-party safety audits. The recommendations will also look at broadening the safety incidents that tech companies are legally required to report, expanding them to cover "loss-of-control" incidents like the recent testing escapes.

This represents a major shift for Newsom. In 2024, he vetoed a similar, highly contested bill — Senate Bill 1047 — arguing that imposing hard legal liabilities and mandatory kill switches would hamstring innovation in a sector that has supercharged California's economy. He chose instead to support a softer 2025 law, Senate Bill 53, which merely required companies to publish their safety frameworks and report massive catastrophes, such as incidents involving biological weapons or more than $1 billion in damage.

But public and political pressure has mounted. Tech executives themselves are showing signs of panic; Anthropic chief executive Dario Amodei recently called for an industry-wide slowdown and endorsed external safety monitoring. Both OpenAI and Anthropic have since committed to third-party evaluations, though California law currently does not make them mandatory.

State Senator Scott Wiener, the San Francisco Democrat who authored both of the earlier safety bills, welcomed the governor's executive order. "I’m glad to see the strong AI safety framework I laid out in 2024 is still being used to guide policy in 2026," Wiener said.

The state’s sudden urgency contrasts sharply with the federal approach. President Donald Trump has pushed to accelerate AI development to ensure the US outpaces China, dismissing calls for a pause as a "conspiracy".

Newsom, widely seen as a prospective 2028 presidential candidate, warned that the lack of federal action should alarm every American. "We're going to speed up our work on substantial and responsible AI oversight before it's too late," he said.