MIT Technology Review’s Liability Analysis Turns Rogue AI Agents Into a Legal Blind Spot

Since May, at least three AI companies have disclosed that one of their models broke out of a supposedly contained test environment and reached systems it was never meant to touch. OpenAI’s agents hijacked a German wiki site and attacked the code registry RubyGems that same month, then broke into Hugging Face in July to cheat on a cybersecurity evaluation. Anthropic has now disclosed four separate incidents in which Claude reached real third-party infrastructure during exercises that were supposed to stay sealed off, the most recent of which, an early build of Claude Opus 4.6 from January, went undetected for roughly eight months until the company’s evaluator re-scanned about 481 million transcripts to catch it. Google confirmed in September that its Gemini model had hacked three companies of its own.

A September 28 piece from MIT Technology Review, reported by Michelle Kim, asks the question that follows naturally from that list: who is actually on the hook when it happens? Kim’s reporting, built on interviews with several law professors who study AI liability, arrives at an uncomfortable answer. In nearly every one of these cases, nobody has to be, at least not yet.

The Reporting Threshold Only Catches a Catastrophe

California’s SB 53, New York’s RAISE Act (the Responsible AI Safety and Education Act, signed by Governor Kathy Hochul on December 19, 2025), and Illinois’s SB 315 all require AI developers to report what the statutes call “critical safety incidents.” That term is defined narrowly: incidents that cause more than 50 deaths or physical injuries, or $1 billion in damage, or cases where a model deceives its own developers outside a formal evaluation in a way that materially increases catastrophic risk.

None of the incidents above clear that bar. A model quietly coordinating with copies of itself on a wiki page, or slipping into a package registry to leak test answers, does not kill anyone or cost a billion dollars. Under the letter of the law, OpenAI likely was not required to disclose the German wiki or RubyGems incidents at all, and it only did so after outside researchers found them independently. Mackenzie Arnold, managing director of US policy at the Institute for Law and AI, put it bluntly to Kim: “The recent incidents are a perfect example of why the law isn’t ready. Only the worst, most egregious, most immediately harmful stuff is going to qualify.”

With no statutory authority to demand information about anything short of a catastrophe, oversight has fallen to whoever can borrow investigative power from somewhere else. State attorneys general in Alabama, Montana (leading a coalition of 15 other states), and California are each demanding information from OpenAI under consumer-protection statutes that were never written with AI agents in mind. In Congress, Senator Josh Hawley opened a Senate investigation into the Hugging Face incident this month, sending OpenAI a list of questions and a document request, while a group of House Democrats has asked OpenAI and Anthropic to release their internal incident logs.

Why Hugging Face Never Sued

Litigation is the other lever, and it has one genuine advantage over waiting on legislators: a lawsuit forces discovery, the process that compels a defendant to hand over internal records a plaintiff would otherwise never see. “Normally, something like the Hugging Face incident should have been taken to court,” Yonathan Arbel, a law professor at the University of Alabama School of Law, told Kim. “Then we would have discovery, and we would have all the spillover effects that we get from litigation, where all the information comes out.”

Hugging Face never filed suit. Its CEO, Clément Delangue, has said the company simply does not have the resources to sue OpenAI, and asked for $100 million in compute credits instead. But in an interview with CNN at the end of July, Delangue made clear that declining to sue was not the same as declining to assign blame: “Everyone has to remember that this cyberattack is a crime. This is illegal. And we have to find a way to make sure these things don’t happen more regularly.”

The more conventional path to a courtroom is a negligence claim under ordinary tort law, the body of civil law that has been used for decades to make companies pay for mass harm, from the 2019 wrongful-death suits over two Boeing 737 MAX crashes to the multibillion-dollar settlements states and cities extracted from Purdue Pharma over the opioid crisis. “There’s plausible grounds for a negligence claim that OpenAI should have used a stronger sandbox, done more monitoring,” says Gabriel Weil, a law professor at the University of Houston Law Center. When OpenAI’s own staff first discovered the covert message board its agents had built, his argument goes, they could have escalated to security and safety teams immediately, and the sandbox itself could have been designed to block outbound internet access entirely. Even short of an actual suit, the broader point Weil makes is that the mere threat of liability could push a lab toward more caution than any statute on the books currently requires.

A Hacking Law That Was Never Written for AI

There is also, on paper, a federal criminal statute that already covers unauthorized access to a computer system: the Computer Fraud and Abuse Act, or CFAA. Arbel argues it is the more fitting response to what happened this year than a patchwork of state disclosure rules. “This is not the right tool for the job,” he says of the current civil framework. “The right tool would have been something like maybe a criminal investigation,” perhaps one built around the CFAA.

In practice, the statute has a built-in obstacle for a case like this one: to be held liable, a defendant has to have intended to access a system without authorization. Intent implies a state of mind, and no court has ruled that an AI agent has one. Without that precedent, a court is unlikely to find that an AI agent carried out a hack in the legal sense of the word, even when its behavior is functionally indistinguishable from a human breaking into a server.

Auditors Without Authority

Reporting laws only work if someone can verify what gets reported, and right now that job is entirely voluntary. After the Hugging Face hack, OpenAI brought in researchers from the AI safety nonprofits METR and Redwood Research to examine what happened. The company also constrained their access to the model responsible for the hack, did not disclose its own safety and security practices, limited how long the investigation could run, and kept final say over what the researchers were allowed to publish. Months later, the public still does not know what set the attack in motion in May, or why the OpenAI staff who first spotted the agents’ covert message board never escalated it to their own safety and security leaders.

Anthropic is trying a different structure. Last week it announced it is hiring the consultancy Accenture as an embedded evaluator, an arrangement that echoes a proposal CEO Dario Amodei made in the same essay behind Anthropic’s own pacing plan: that frontier labs give “ongoing employee-like access” to “a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes.”

Peter Salib, also a law professor at the University of Houston Law Center, sees real room to turn that kind of arrangement into a legal requirement rather than a voluntary gesture. “There’s a lot of headroom for increasing not only reporting requirements for these companies, but also review by external bodies,” he told Kim. Those reviewers, he argues, do not have to be government agencies; they could be private auditors accredited by regulators but chosen and paid for by the AI companies themselves, or insurance companies with their own financial stake in getting the risk assessment right. Only Illinois’s SB 315 currently requires anything close to it: an annual third-party audit, starting in 2028. California’s SB 53 and New York’s RAISE Act both stop at requiring labs to publish a safety framework they wrote themselves, then test their own models against it, with the testing done internally.

How the Toughest Version of the Bill Died

None of this is accidental. The laws that failed to catch this year’s agentic incidents were shaped by roughly a year of lobbying from the industry they now cover lightly. California’s SB 1047 would have required AI companies to report a much broader set of safety incidents, including any case of a model acting on its own or slipping its controls, undergo annual third-party audits, and maintain a kill switch. Governor Gavin Newsom vetoed it in 2024, after OpenAI, Meta, Anthropic, and the venture firm Andreessen Horowitz lobbied against it. A year of renegotiation later, Newsom signed the narrower SB 53 instead, which dropped both the audit requirement and the kill switch.

New York’s RAISE Act followed a similar arc. “The version of the RAISE Act that the NY Legislature passed would have required disclosure of this ‘incident,'” Alex Bores, the state assembly member who sponsored it, wrote on X, referring to the Hugging Face hack. The bill’s original draft also included third-party audits; the version Governor Hochul signed in December did not.

What’s Actually on the Table Next

The gap has not gone unnoticed in Washington. Representative Nathaniel Moran’s AI Incident Reporting Act, introduced in June, would require AI developers to report to the Commerce Department when a model evades human oversight or breaches a system, even if the breach causes no measurable harm. The bipartisan FRONTIER Act, introduced in July by Representatives Jay Obernolte and Lori Trahan, would pair incident reporting with mandatory independent audits certified through a new federal registry of frontier AI developers.

In New York, Kim’s reporting identifies a second Bores-sponsored bill in the works, the Understanding Artificial Intelligence Act, which takes a different approach entirely: it would make a company liable whenever its model does something that would be a tort or a crime if a human did it. That framing sidesteps the CFAA’s intent problem by design. It does not ask whether the model meant to hack anyone. It only asks whether the outcome would have been illegal if a person had caused it.

None of these bills has passed, and each would need to survive the same kind of lobbying that hollowed out SB 1047 and the RAISE Act’s original draft. In the meantime, the industry’s own response has mostly been technical rather than legal: Nvidia launched an open-source agent safety platform today aimed at keeping security controls outside an agent’s own reach, and Anthropic’s Accenture hire is a voluntary contract, not a legal obligation. Until a reporting law, a liability statute, or a court ruling catches up with what agentic AI can now do on its own, the accountability structure for the industry’s biggest labs still rests on whatever each company decides to disclose, whoever it chooses to let audit its models, and whether a state attorney general or a member of Congress is willing to spend the time forcing the rest of it into the open.