New York passes a bill to prevent AI-fueled disasters

[ad_1] New York state lawmakers handed a invoice on Thursday that goals to stop frontier AI fashions from OpenAI, Google, and Anthropic from contributing to catastrophe situations, together with the demise or harm of greater than 100 folks, or greater than $1 billion in damages.

The passage of the RAISE Act represents a win for the AI security motion, which has misplaced floor lately as Silicon Valley and the Trump administration have prioritized velocity and innovation. Security advocates together with Nobel laureate Geoffrey Hinton and AI analysis pioneer Yoshua Bengio have championed the RAISE Act. Ought to it develop into legislation, the invoice would set up America’s first set of legally mandated transparency requirements for frontier AI labs.

The RAISE Act has among the identical provisions and objectives as California’s controversial AI security invoice, SB 1047, which was in the end vetoed. Nevertheless, the co-sponsor of the invoice, New York State Senator Andrew Gounardes, informed TechCrunch in an interview that he intentionally designed the RAISE Act such that it doesn’t chill innovation amongst startups or educational researchers — a standard criticism of SB 1047.

“The window to place in place guardrails is quickly shrinking given how briskly this expertise is evolving,” stated Senator Gounardes. “The those who know [AI] one of the best say that these dangers are extremely doubtless […] That’s alarming.”

The RAISE Act is now headed for New York Governor Kathy Hochul’s desk, the place she may both signal the invoice into legislation, ship it again for amendments, or veto it altogether.

If signed into legislation, New York’s AI security invoice would require the world’s largest AI labs to publish thorough security and safety studies on their frontier AI fashions. The invoice additionally requires AI labs to report security incidents, akin to regarding AI mannequin habits or dangerous actors stealing an AI mannequin, ought to they occur. If tech firms fail to reside as much as these requirements, the RAISE Act empowers New York’s lawyer common to convey civil penalties of as much as $30 million.

The RAISE Act goals to narrowly regulate the world’s largest firms — whether or not they’re primarily based in California (like OpenAI and Google) or China (like DeepSeek and Alibaba). The invoice’s transparency necessities apply to firms whose AI fashions had been skilled utilizing greater than $100 million in computing assets (seemingly, greater than any AI mannequin out there in the present day), and are being made out there to New York residents.

Whereas much like SB 1047 in some methods, the RAISE Act was designed to deal with criticisms of earlier AI security payments, in line with Nathan Calvin, the vp of State Affairs and common counsel at Encode, who labored on this invoice and SB 1047. Notably, the RAISE Act doesn't require AI mannequin builders to incorporate a “kill swap” on their fashions, nor does it maintain firms that post-train frontier AI fashions accountable for vital harms.

However, Silicon Valley has pushed again considerably on New York’s AI security invoice, New York state Assemblymember and co-sponsor of the RAISE Act Alex Bores informed TechCrunch. Bores known as the trade resistance unsurprising, however claimed that the RAISE Act wouldn't restrict innovation of tech firms in any method.

“The NY RAISE Act is one more silly, silly state stage AI invoice that may solely damage the US at a time when our adversaries are racing forward,” stated Andreessen Horowitz common companion Anjney Midha in a Friday publish on X. Andreessen Horowitz and startup incubator Y Combinator had been among the fiercest opponents to SB 1047.

Inside baseball coverage thread: Final evening, NY handed the RAISE act, which might set up some transparency necessities for frontier fashions. We @anthropicai haven’t taken a place on this invoice. However I believed it’d be useful to provide some extra context:

— Jack Clark (@jackclarkSF) June 13, 2025

Anthropic, the safety-focused AI lab that known as for federal transparency requirements for AI firms earlier this month, has not reached an official stance on the invoice, co-founder Jack Clark stated in a Friday publish on X. Nevertheless, Clark expressed some grievances over how broad the RAISE Act is, noting that it may current a danger to “smaller firms.”

When requested about Anthropic’s criticism, state Senator Gounardes informed TechCrunch he thought it “misses the mark,” noting that he designed the invoice to not apply to small firms.

OpenAI, Google, and Meta didn't reply to TechCrunch’s request for remark.

One other widespread criticism of the RAISE Act is that AI mannequin builders merely wouldn’t supply their most superior AI fashions within the state of New York. That was an identical criticism introduced towards SB 1047, and it’s largely what’s performed out in Europe due to the continent’s powerful laws on expertise.

Assemblymember Bores informed TechCrunch that the regulatory burden of the RAISE Act is comparatively mild, and due to this fact, shouldn’t require tech firms to cease working their merchandise in New York. Given the truth that New York has the third largest GDP within the U.S., pulling out of the state isn't one thing most firms would take calmly.

“I don’t wish to underestimate the political pettiness which may occur, however I'm very assured that there isn't a financial cause for [AI companies] to not make their fashions out there in New York,” stated Assemblymember Bores.