California Governor Gavin Newsom vetoed a first-of-its-kind state bill that would potentially enact the most impactful artificial intelligence regulation in the country.
The measure, known as the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act, would require safety measures from companies that spend more than $100 million to train AI models. It aims to prevent potential harms by AI, such as mass casualty events, and includes implementing a "kill switch" to completely shut down a rogue model.
California is home to some of AI's biggest players, including OpenAI, Anthropic, Google (GOOG), and Meta (META). However, in his veto message on Sunday afternoon, Newsom said SB 1047 is "well-intentioned," but it "does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data. Instead, the bill applies stringent standards to even the most basic functions — so long as a large system deploys it. I do not believe this is the best approach to protecting the public from real threats posed by the technology.”
Along with the veto, Newsom announced he's working with leading experts — including "the godmother of AI" Fei-Fei Li — to set guardrails around the deployment of generative AI. He also ordered state agencies to expand their assessment of the risks associated with the technology.
Regulating the technology has been a flash point in Silicon Valley and beyond. OpenAI, Google, and Meta publicly opposed the bill. Anthropic, backed by Amazon (AMZN), cautiously supported it after suggesting amendments to its original version.
Despite the pushback from Big Tech, more than 100 current and former employees from Google, Meta, OpenAI, and Anthropic called on Newsom to sign the legislation earlier this month, expressing concerns that “the most powerful AI models may soon pose severe risks.”
More than 125 Hollywood actors, directors, and entertainment leaders had also urged Newsom to sign the bill, writing in a letter: "We fully believe in the dazzling potential of AI to be used for good. But we must also be realistic about the risks."
SB 1047 had to thread the needle between encouraging innovation in the rapidly changing industry while ensuring the tech is used responsibly.
Newsom discussed his concerns over SB 1047 with Salesforce (CRM) CEO Marc Benioff at the annual Dreamforce conference earlier this month. "The impact of signing wrong bills over the course of a few years could have a profound impact," Newsom said referring to the state's competitiveness.
"This is a space where we dominate, and I want to maintain our dominance. I want to maintain our innovation. I want to maintain our ecosystem. I want to continue to lead. At the same time, you feel a deep sense of responsibility to address some of those more extreme concerns that I think many of us have, even the biggest and strongest promoters of this technology have."
Supporters of the legislation include the "Godfathers of AI," Yoshua Bengio and Geoffrey Hinton, and billionaire tech CEO Elon Musk, who owns a large AI model company, xAI.
California's artificial intelligence bill - SB1047.
"The capabilities of AI models are increasing at a very rapid pace, and so are the sums invested to reach the level of Artificial General Intelligence and beyond," Bengio told Yahoo Finance. "The pursuit of this stated goal by a handful of major companies, racing against each other without the proper guardrails, entails major risks for our societies and democracies."
The main author of the bill, California state Senator Scott Wiener, has said it's a reasonable framework for an underregulated technology. Wiener has been vocal about the need for a strong federal law that would set nationwide guardrails for all developers.
However, Wiener isn't hopeful that a national AI safety bill will be a reality anytime soon, calling Congress "completely paralyzed when it comes to technology policy" at a press conference last month.
"Let me be clear — I agree with the author — we cannot afford to wait for a major catastrophe to occur before taking action to protect the public," Newsom wrote, adding, "I do not agree, however, that to keep the public safe, we must settle for a solution that is not informed by an empirical trajectory analysis of Al systems and capabilities. Ultimately, any framework for effectively regulating Al needs to keep pace with the technology itself."
The bill was set to reshape the future of AI and numerous tech executives have spoken out.
"There are risks associated with AI, less to do with the models themselves and more to do with what the models are allowed to do in the real world if left completely unsupervised," Affirm (AFRM) CEO Max Levchin told Yahoo Finance at the Goldman Sachs Communacopia and Tech Conference.
"So I'm not diminishing or dismissing the need for controls, and model governance and oversight and thoughtful rulemaking. I would just not want to 'shut it all down' to quote another AI doomerist."
Although the bill passed the state's Assembly 48-16 (seven Democrats voted no) and Senate 30-9 (one Democrat voted no) in August, it's faced some political opposition from California Democrats.
Critics of SB 1047 include eight California house members — Ro Khanna, Zoe Lofgren, Anna Eshoo, Scott Peters, Tony Cárdenas, Ami Bera, Nanette Diaz Barragán, Lou Correa — and longtime Newsom ally former House Speaker Nancy Pelosi.
In the last month, Newsom has signed 17 AI-related bills aimed at combating deepfake election content, protecting actors and entertainers over their digital likenesses, and regulating sexually explicit content created by AI, among other measures.
The new set of laws will require developers and social media companies to prevent irresponsible use of their platforms and the use of deceptive content.
While this legislation addresses the immediate dangers of AI, SB 1047 looks ahead at some of the most extreme risks posed by advanced models.
While speaking in front of the United Nations General Assembly on Tuesday, President Joe Biden called on world leaders to establish AI standards that protect human life.
"This is just the tip of the iceberg of what we need to do to manage this new technology," President Biden said. "In the years ahead, there may well be no greater test of our leadership than how we deal with AI."
Yasmin Khorram is a Senior Reporter at Yahoo Finance. Follow Yasmin on Twitter/X @YasminKhorram and on LinkedIn. Send newsworthy tips to Yasmin: [email protected]