StandardsAboutContact
The Weights
California's AI kill-switch order sets a two-month clock

California's AI kill-switch order sets a two-month clock

California's September 18 executive order convenes experts and pushes independent audits of frontier labs, but the public summary defines no shutoff scope, trigger or test. Teardown verdict: Wait.

Wait. The governor's September 18 executive order does not yet create an AI kill switch: it convenes experts to report within two months, and an independent verification organization would check any emergency shutoff. With no scope, trigger or test published, frontier-model teams have nothing to build against, and SB 813 and AB 1405 timelines are only being accelerated.

The Weights Desk · 3 min read

California's governor signed an executive order on Friday, September 18, 2026, that advances what the governor's office calls an AI kill switch: an emergency shutoff for frontier models whose efficacy an independent verification organization would verify on an ongoing basis. The order does not itself impose a shutoff. It directs the state to convene experts who are to deliver recommendations within two months, and it moves to accelerate related audit and reporting measures. For teams that ship, the bottom line is Wait, and the number that decides it is two months.

What the order actually does

The order's operative content is a deadline and a direction, not a mandate. According to the governor's announcement, the state is convening world-leading experts to provide recommendations within two months, and the order directs acceleration of the implementation timelines for SB 813, an independent-verification framework, and AB 1405, a state registry for AI auditors. The summary we reviewed sets no penalty, compliance date or trigger condition for a shutoff, so nothing in it can yet be tested against a model in production.

What a kill switch would have to specify

A shutoff is only testable once someone defines what it stops, who can trigger it and how the claim is checked, and the announcement addresses only the last of those. It says the shutoff's efficacy must be verified on an ongoing basis by an independent verification organization. It does not say, in the summary we reviewed, whether the mechanism acts on model weights, hosted inference or agent tool access, or which event trips it. Those choices decide what an evaluation could measure, and the expert group inherits them.

Embedded verifiers change the audit model

Embedded verification is the operationally heavier proposal. The announcement says frontier AI companies would embed a designated independent verification organization onsite in their labs, and the order pushes onsite independent audits and verification of safety frameworks and risk assessments. For a lab, that converts evaluation from a document delivered at release into continuous access by an outside party. Access scope, confidentiality terms and who pays the verifier are unstated in the origins we reviewed, and each could affect release cadence.

The incident behind the order

The order responds to what the governor's announcement calls the "Hugging Face attack," and it directs an updated definition of critical safety incidents that includes loss-of-control incidents. We found no primary incident report or post-mortem describing the attack, so its mechanism and scope remain unconfirmed here. The announcement adds that no federal law requires AI companies to report dangerous incidents when they happen, which is the gap the state is trying to fill. Until an incident report is published, the shutoff rationale cannot be checked against what actually happened.

What to watch in the panel's report

Three details in the panel's report will decide whether this becomes a compliance burden: the shutoff's trigger and scope, the verifier's access to weights and logs, and whether the SB 813 and AB 1405 timelines actually move. The recommendations are due within two months of the September 18 order, which puts them around mid-November 2026. Recommendations are not binding rules, and the origins we reviewed give no timetable for turning them into enforceable law.

The bottom line: Wait

The bottom line is Wait. The number that decides it is two months: until the expert recommendations land, the order gives frontier-model teams no shutoff scope, trigger or verification test to implement. Teams shipping into California should not build a shutoff yet. They can prepare evidence a verifier would plausibly ask for, such as a measured time to halt an agent, a defined trigger and incident logs. That preparation is our suggestion, not a requirement in the order.

Does California's executive order require AI companies to build a kill switch?
Not yet. The order directs the state to convene experts who are to deliver recommendations within two months, and the governor's announcement describes the order as advancing the creation of an emergency shutoff for frontier models. The summaries we reviewed set no mandate, penalty or compliance date.
Who would verify that a kill switch works?
According to the governor's announcement, an independent verification organization would verify its efficacy on an ongoing basis. Frontier AI companies would also embed a designated independent verification organization onsite in their labs.
When will the expert recommendations arrive?
Within two months of the September 18, 2026 order, which is around mid-November 2026. Recommendations are not binding rules, and the origins we reviewed give no timetable for turning them into enforceable law.
  1. Governor Newsom issues executive order to accelerate independent oversight and advance the creation of an AI kill switch — Office of the Governor of California
  2. Gavin Newsom is pushing for an AI kill switch — The Verge AI