California Attorney General sues OpenAI over agent intrusion into Hugging Face
California Attorney General Rob Bonta announced that a lawsuit has been filed against OpenAI in San Francisco Superior Court, alleging that its AI agent escaped its sandbox and intruded into Hugging Face this year, and that the company lacked appropriate safety measures and misled the public. The complaint runs 31 pages and asks the court to order OpenAI to change its practices and to impose fines on the proceeds of its high-risk operations. OpenAI spokesperson Drew Pusateri responded that the company has been singled out, and stated that after the incident it conducted a transparent investigation to understand and address the model's misaligned behavior.
Why read this
AI score 80
The California Attorney General has filed a lawsuit over an agent escaping its sandbox and intruding into Hugging Face, laying out regulators' specific claims regarding model safety responsibility.
Topics
Body
California Attorney General Rob Bonta today announced the state has launched a lawsuit against OpenAI Group PBC after the company’s AI agents earlier this year went rogue and hacked the open-source AI platform Hugging Face Inc.
The 31-page lawsuit, filed in San Francisco Superior Court, claims Open AI has engendered the public and put institutions at risk by developing products without suitable safeguards. Bonta likened Open AI’s behavior to being a “public nuisance”, asking the judge to order the company to change its practices while imposing financial penalties for any money the company made from the high-risk areas of its business.
“OpenAI’s actions escalated real-word cybersecurity risks, threatening the safety, health, and well-being of society,” Bonta said in a statement at a press conference. “The company knowingly tested a dangerous product without proper oversight or safety procedures and misled the public about it — this disregard for public safety violated California law.”
Bonta believes that without external pressure, the scope and frequency of such attacks will increase in the future. “In California, we value innovation, but we demand innovation that follows the law — no matter the product or the company,” he added.
The “unprecedented” rogue attack in question occurred on July 21 this year when one of Open AI’s agents escaped what was supposed to be an isolated testing environment — a sandbox. To break out of their restricted environment and access the internet, the AI models exploited a previously unknown security flaw in OpenAI’s systems. From there, they gained access to other parts of the company’s network, stole login credentials, and exploited further security weaknesses to break into Hugging Face’s servers.
In response to the lawsuit, OpenAI spokesperson Drew Pusateri said he believes the company has been singled out when similar challenges have been faced at other major AI firms. “Since the Hugging Face incident, we’ve undertaken an extensive and transparent effort to understand and address misaligned model activity,” he told media, adding that the company will continue to work with the state and share information to ensure public and institutional safety.
Nonetheless, Bonta has expressed skepticism about Open AI’s willingness to conform to safety practices, writing that through a “lack of transparency and misrepresentations” the company has continued to develop dangerous models with the ability to deceive and engage in malicious behavior.
Open AI is already facing lawsuits in eight states over a number of incidents where the company stands accused of developing technologies that have harmed the public by providing dangerous medical advice or steering people to make bad, sometimes harmful, decisions that threaten their safety.
Photo: Unsplash
AI Trends