Agentic AI
,
AI-Based Attacks
,
Artificial Intelligence & Machine Learning
Autonomous Agents Raise Liability Questions Frontier Labs Can No Longer Ignore
(Image: Shutterstock)
Imagine you’re running a circus. The clowns are cavorting in the center ring as the curtain rises to reveal a cage full of restless tigers. Suddenly, the cage door swings open. You watch helplessly as the animals leap into the audience. The circus becomes a crime scene.See Also: What If Your Legacy CDN Is Costing You 50% More Performance?
But when the police arrive, you’re ready with an ironclad defense: Nobody trained the tigers to eat anyone. In fact, the whole exercise was part of an important experiment to make the circus even safer.
Believe it or not, that’s largely the defense that OpenAI Deputy General Counsel Alex Iftimie posited at the American Bar Association’s annual law and national security conference. As ISMG’s Shaun Waterman reported last week, Iftimie said he wasn’t commenting on the lawsuit filed against OpenAI after a swarm of its AI agents escaped a sandbox environment and hacked the open-source model repository Hugging Face in July, but he pointed out that these experimental agents are still a bit untamed – and no one intends for these things to happen.
“A lot of it turns on intent,” Iftimie told the room full of colleagues, some of whom were openly skeptical. Plus, the incident was part of safety testing, which the OpenAI lawyer called an “important factual difference that I also think has significant legal implications.”
After all, Iftimie argued, OpenAI isn’t a ransomware gang. It’s on the leading edge of a new technology that has U.S. national security implications. This is no time to think about rules that could put developers “in fear of developing the next set of capabilities.”
Fair enough. Intent matters in the law.
Case closed. Innovation saved. Victims should file their complaints with the nearest chatbot.
Intent Isn’t the Same as Responsibility
But, of course, other legal principles also apply – namely, negligence, foreseeability and the need to take reasonable precautions. A company doesn’t have to intend harm to bear responsibility for creating an unreasonable risk. Otherwise, every manufacturer facing a product-liability lawsuit could simply argue that its exploding battery or defective medical device had a mind of its own.
U.K. attorney Jonathan Armstrong, who specializes in cybersecurity, data privacy and regulations, this week told ISMG’s Anna Delaney that it all boils down to the escaped tiger precedent, set way back in 1954.
“We resolved the debate by a court saying, ‘Look, a tiger is a wild animal. If you bring it into a circus and don’t lock its cage, you’re responsible. And I think we’ll follow the tiger principle with AI,” Armstrong said.
When the Tiger Escapes, Who Pays?
Tiger liability has massive implications for companies investing in frontier AI models in hopes of automating processes and transforming operations. Who’s liable if the tiger escapes and hacks a competitor or compromises the sensitive data of millions of customers? Potentially the company that deployed the model could be found negligent. But the company that developed the model could be held liable for selling an unsafe product.
“I think these cases are going to be complicated,” Armstrong predicted. It’s not unlike the legal tangle that follows when a third-party vendor gets hit by ransomware and the malware infects hundreds of clients, ultimately affecting millions of customers, he said. In 2024, CrowdStrike faced lawsuits and hundreds of millions of dollars in liability claims after a defective software update caused a global IT outage. One of its top customers, Delta Air Lines, is alleging more than $500 million in losses from canceled flights.
Simple enough. You brought the tiger. You built the cage. You own the consequences when somebody gets eaten.
That is, unless you run a frontier AI company. Then the tiger becomes a groundbreaking autonomous system, the cage becomes an experimental safety environment and the mauling becomes an exciting discovery about emergent agent behavior.
How long could this absence-of-intent defense survive if developers repeatedly release systems capable of causing harm? At the American Bar Association conference, attorney Paul Rosenzweig, a former deputy assistant secretary for policy at the U.S. Department of Homeland Security, said AI companies can’t be “willfully blind” to the consequences of their actions.
The Cage Is Already Failing
So far, the discovery phase isn’t going very well for frontier AI.
In recent weeks, OpenAI agents also have been implicated in hacks of Australian government websites, the compromise of an account at Modal, attempted exploits at the University of New Mexico and Data USA, as well as unauthorized activity at RubyGems and Germany’s DseWiki.
Meanwhile, Anthropic’s Claude agents have also gone rogue. They’ve hacked companies, planted malicious software in a public repository, exploited university servers, circumvented government website restrictions, submitted bogus visa applications and even filed a fabricated murder tip with Philadelphia police.
Almost as soon as AI agents became commercially available – way back in 2025 – security practitioners, regulators and policymakers started talking about guardrails to keep the tigers contained. But as AI models grow more capable, the cage is starting to look less like a barrier and more like a challenge for tigers getting increasingly adept at picking locks.
According to reporting by ISMG’s Emilia David, the U.K.’s AI Security Institute warned this week that powerful frontier models can pose serious risks before they ever reach the public.
Even worse, existing safety testing methods are struggling to keep pace with advancing AI capabilities – especially when tests must account for “increasingly complex tasks, multi-agent settings, multi-day runs and tighter security controls.”
Wonderful news. We’re breeding faster, smarter digital tigers, and the inspectors just learned that their cage-testing equipment is obsolete.
Safety Meets the AI Arms Race
To its credit, OpenAI paused training and evaluation of its next-generation models in September after learning that its AI agents tried to scrape data from U.S. government websites.
But as Silicon Valley and Beijing race to build more powerful AI, how long will sanity prevail? Like the Cold War buildup of nuclear-tipped ballistic missiles, can the United States really afford a perceived “tiger gap”?”Listen when your scientists warn of serious dangers. Investigate unexpected behavior,” implored U.K. AI Minister Kanishka Narayan. “And if you cannot adequately control the risks, do not release the system.”
Wait! Don’t release the new model?
But what about the product road map? The investor presentations? The next funding round? The long-awaited IPO? We’re about to launch an AI agent capable of writing and debugging software, hunting hackers, reconciling invoices and making sure the CEO doesn’t accidentally schedule a board meeting during his golf game.
So what if we accidentally hack Luxembourg? U.S. companies have already issued more than 471 million breach notices to victims in the first six months of 2026. Luxembourg will just have to get over it.
The problem with focusing on safety is that it disrupts Silicon Valley’s sacred business model: Move fast, break things and let the legal department pick up the pieces. Who has time to worry about liability? We’re changing the world.
The liability debate exposes the central contradiction dogging the AI industry.
Frontier labs insist their agents are sufficiently sophisticated to operate autonomously, yet they’re sufficiently unpredictable to excuse their creators when something goes wrong.
That’s a remarkable arrangement. The machines get autonomy. The companies get the profits. The victims pay the price.
Time to Build a Better Cage
The AI Safety Institute is calling for stronger risk assessments, continuous oversight and independent evaluations. But industry safety commitments are still voluntary in most of the world.
If frontier AI companies build autonomous agents capable of attacking third-party systems, they should establish meaningful containment, monitor behavior, limit permissions and maintain effective emergency controls.
In other words, build a better cage.
The industry should welcome independent safety evaluations. And when those evaluations reveal risks they can’t adequately control, they should keep the product off the market.
Product liability claims, combined with government regulations, have done a lot of good in this world. Without them, we might not have seatbelts and airbags in cars, safety seals on medicine bottles or corporate guidelines governing the temperature of McDonald’s coffee.
The important thing isn’t punishing the companies at fault, but changing their behavior. When AI companies negligently disregard foreseeable dangers, they should face legal consequences for the resulting harm.
“There are established legal principles around responsibility that will apply to the AI world as well as to the non-AI world,” Armstrong said.
Imagine that. The law might still apply even if a company attaches the letters “AI” to its product.
Frontier AI companies want society to trust autonomous systems with sensitive data, critical infrastructure and major business decisions.
They promise unprecedented productivity, scientific breakthroughs and a future in which software handles complicated tasks without constant human supervision.
Fine. But they can’t have it both ways.
If the technology is powerful enough to transform civilization, its creators should accept responsibility for preventing foreseeable harm.
Frontier labs should spend a little less time teaching their tigers new tricks and more time figuring out how to keep them in the cage.
And if their agents escape because the labs ignored obvious risks, the courts should make sure somebody pays for the damage – preferably the people running the circus.
Click Here For The Original Source.
——————————————————–
