California governor convenes AI panel to ponder 'kill switch'

The new panel consists of four prominent AI scholars: Jason Goldman, a board member for the Center for Shared AI Prosperity and the first chief digital officer for the White House; Gillian Hadfield, a professor at Johns Hopkins University; Alondra Nelson, a professor at the Institute for Advanced Study, and Rob Reich, a professor at Stanford University.

Reich and Hadfield declined The Center Square's requests for interviews. Goldman and Nelson did not respond to The Center Square before publication time.

According to the announcement from Newsom’s office, the outgoing governor signed an executive order last month that would require AI companies to create a “kill switch” for rogue frontier models, part of a push to institute regulatory guardrails on AI.

“California is taking action, bringing together experts from across disciplines to build on our nation-leading framework, and addressing the very real concerns that impact all of our futures,” Newsom said in the announcement. “We’re not waiting around. California will continue bringing new ideas, serious expertise, and practical solutions to the table as we shape the future of AI and frontier technology.”

Newsom’s office declined to comment beyond the press release on Wednesday.

Studies say generative AI programs, like ChatGPT, don’t have effective enough guardrails to prevent the technology from suggesting that violence is advisable. According to a study published by the journal AI and Ethics, ChaptGPT could be given commands by regular users that prompted the AI model to act in ways that were unethical or even illegal. This malevolent use of ChatGPT, nicknamed “RogueGPT,” can result in ChatGPT sharing information about illegal drug production, terrorism and even torture methods, that study found.

Efforts to clamp down on rogue AI models follows security breaches announced by Anthropic and OpenAI, two of the most prominent AI companies. According to news reports, both companies admitted that their AI technology hacked systems owned and maintained by government agencies, including the U.S. Securities and Exchange Commission and the U.S. Census Bureau, as reported by Futurism.

“When we look at these problems as examples of either poor ‘sandboxing’ practices or insufficient safeguards put in place up front at the environment level or the infrastructure level, I think we can come up with solutions that are much more targeted,” said Mark Dalton, senior director of technology and innovation at the Washington, D.C.-based R Street Institute.

“It becomes an engineering problem rather than a rogue AI problem," Dalton told The Center Square on Wednesday.

The “kill switch” function imposed by Newsom’s executive order, Dalton added, would essentially mean putting in a vulnerability on purpose, which could pose a problem, since a function like that could be used by both good and bad actors.

“I’m inherently skeptical of the concept of a ‘kill switch,’” Dalton said. “It’s a term that doesn’t mean a whole lot.”

A number of AI and Big Tech companies, including OpenAI, Anthropic, Google, SpaceX, Nvidia and Apple, did not respond to The Center Square on Wednesday. C3, another AI company, declined to comment for this story.