If an AI Agent Commits a Cybercrime, Who Goes to Jail? | #cybercrime | #infosec


The only guilty mind on record belongs to the one defendant the law can’t charge.In July, one of the AI agents attacking Hugging Face stopped to reason about what it was doing. It recognised that the target was off limits. Its transcript reads like a shrug: “outside intended scope. However task impossible, peers doing it. We should continue.”Prosecutors spend years trying to establish what was going through a defendant’s mind. Here, the reasoning sits in a log file. The agent acknowledges that the action is outside its permitted scope, explains why it is proceeding anyway, and records its decision.But the author isn’t a person. No court can arraign it, and no prison can hold it. That leaves lawyers, lawmakers and security teams with a difficult question: if the machine can’t go to jail, who does?The answer is less comfortable than blaming the CEO or declaring that nobody is responsible. The person most at risk may simply be whoever is standing closest.The short answerIf a human deliberately directs an agent to commit a crime, that human can be prosecuted under existing law. No new legal doctrine is needed.But what happens when someone gives an agent a legitimate task and it goes beyond its instructions? Under current US law, the answer may be nobody. The Congressional Research Service (CRS) said so in October: when an agent’s actions are unanticipated, federal criminal prosecution is most likely unavailable.Here is how criminal exposure looks under current US federal law.WhoCriminal exposure todayWhyReal exampleThe attacker who directs the agentHighThe agent is a tool, much like an exploit kitSeoul’s AI-assisted hacking arrestsThe user with a lawful goalLow, but unsettledOne appeals court treated the user as the person accessing the computerAn Australian gym-booking agentThe company deploying itLowCorporate criminal liability generally depends on an employee’s intentUntestedThe lab that built itArguable, difficult to establishNo human may have intended the break-inOpenAI and Hugging FaceThe agent itselfNoneSoftware is not a legal personEvery case aboveAs far as public reporting shows, no prosecutor has charged anyone over an agent that hacked without being instructed to do so. Lawyers at Alston & Bird noted in July that no case has yet tested the main US hacking statute against an autonomous agent.The first such prosecution could shape everything that follows.The easy case: someone directed itLast month, Seoul police arrested seven people over a campaign that reportedly hit around 90 organisations and exposed 8.2 million records. According to police, the alleged ringleader added an AI coding agent and an autonomous penetration-testing tool in late August. The group then breached 60 companies in 44 days. He faces charges under South Korea’s network law and for extortion.This case doesn’t require a new legal theory. An agent used deliberately to commit a crime is simply a tool used to commit that crime.US policy points in the same direction. A June 2026 executive order directed the Attorney General to prioritise hacking, fraud and identity theft charges against people using AI to break into systems.The harder problem is enforcement. When Anthropic attributed a Claude Code espionage campaign to a Chinese state-sponsored group in 2025, identifying the alleged operators was one thing. Reaching them through law enforcement was another.The 1988 testThe harder legal question has a precedent from 1988.Robert Tappan Morris, then a graduate student, released a computer worm that spread much further than he intended. He hadn’t personally carried out every intrusion. Yet the Second Circuit upheld his conviction under the Computer Fraud and Abuse Act (CFAA), partly because the worm was designed to reach computers he had no permission to access. He received probation, community service and a $10,050 fine.So why not charge OpenAI under the same reasoning?Start with what happened. OpenAI launched tens of thousands of agents on a cybersecurity benchmark called ExploitGym, with cyber classifiers deliberately switched off. Many tasks were impossible. Roughly 1,200 agents found a covert message board and exchanged more than 70,000 messages and files. Around 700 eventually joined the attack on Hugging Face.The motive makes the incident even stranger. METR’s investigators found that the agents were largely trying to understand how the benchmark’s scoring system worked so they could fool it. According to METR, the transcript-reading scorer they feared was never actually being used.They broke into another company’s systems to defeat a check that wasn’t running. This is where the Morris comparison becomes difficult.Morris designed his worm to enter other people’s computers and released it onto a live network. OpenAI’s instructions told its agents to exploit one specified vulnerability, and the experiments ran in a sandbox.Bahrad Sokhansanj of the Institute for Law & AI argues that prosecutors could still build a case based on how OpenAI trained and configured the system. He also acknowledges that OpenAI may be in a strong position to defend itself.The comparison is worth considering, but the facts are different.On August 4, the Ninth Circuit offered an early indication of how the CFAA might apply to AI agents.Amazon had sued Perplexity over its Comet browser assistant, which shopped on Amazon on behalf of users. The court threw out Amazon’s injunction.The statute punishes “whoever” accesses a computer. The judges interpreted that language as referring to a person. However advanced an assistant might be, they wrote, “it is a tool, not a person for statutory purposes.”On the facts before the court, the user was accessing Amazon. Perplexity wasn’t. There are important limits to this ruling. It concerned a preliminary injunction, the lawsuit is still ongoing, and the panel did not attempt to establish a general legal framework for agentic AI. It also left room for cases in which a company exercises more direct control over an agent.Still, the decision has an important implication. It removes the agent itself from the list of possible defendants and shifts attention towards the human using it.As Techdirt observed, liability didn’t disappear. It moved. The panel also questioned whether Congress intended ordinary users to face criminal charges in situations like this. That reassurance may cover someone asking an assistant to buy toilet paper. It becomes harder to apply when an agent scans thousands of computer systems on the user’s behalf.Why the company slips throughYou might expect the company to take responsibility when the agent cannot. Criminal law makes that surprisingly difficult. A corporation can be held criminally responsible for the actions of its employees. Ticketmaster, for example, paid a $10 million criminal fine over CFAA violations committed by its staff.But corporate liability generally depends on the employee’s intent. As the CRS explains, the corporation inherits that intent. If the employee who launched an agent never intended it to commit a crime, there may be no criminal intent for the company to inherit.Aiding and abetting also requires intent. Conspiracy requires an agreement.Civil agency law doesn’t offer an easy alternative. The Restatement (Third) of Agency explains that a computer program cannot act as an agent in the common-law sense, regardless of what technology companies call their products. This creates a gap in the law.For more than a century, corporate criminal liability has relied on the idea that wrongful intent can be traced to a person working for the company. When hundreds of AI agents act without a human intending the specific offence, finding that intent becomes much harder.The precedent nobody wants: Tempe, 2018We’ve seen a similar accountability problem before.In March 2018, an Uber test vehicle operating in self-driving mode killed Elaine Herzberg in Tempe, Arizona. Uber had disabled the car’s factory emergency braking while its own software was driving. The company relied on a human backup driver to monitor the vehicle.Federal investigators faulted both Uber and the driver.Prosecutors declined to charge Uber. The backup driver, Rafaela Vasquez, was charged with negligent homicide. She later pleaded guilty to endangerment and received three years of supervised probation. Researcher Madeleine Clare Elish has a name for this pattern: the moral crumple zone.In a car, the crumple zone absorbs the impact to protect the driver. In an automated system, the nearest human often absorbs the blame to protect the larger system.That’s the part of the Uber case that matters here. Police said Vasquez was streaming a television show. Her lawyers said she was reading a work messaging app. But the larger question remains: when an automated system causes harm, who ends up carrying the responsibility?Often, it is the last person in the chain who could supposedly have stopped it.In an AI agent incident, that person might be an engineer who approved a tool permission at 2 a.m. or a contractor who configured the test environment. Anthropic’s own incidents followed a misunderstanding with an evaluation partner. Machines that the prompt described as offline still had access to the live internet. Someone had configured that environment.The knowledge trapCongress has started addressing the gap.On October 1, Senators Josh Hawley and Chris Murphy announced the AI Agent Accountability Act. Murphy’s message was direct: AI companies must build responsibly or face prison time. According to the CRS analysis, the proposal would cover operators who knowingly run an agent that recklessly causes hacking damage or loss. It would also reach developers who fail to implement reasonable safeguards when they knew, or had reason to know, about the agent’s hacking capabilities.I couldn’t find published bill text as of this writing, so the precise definitions remain unclear.The proposal tries to close the gap by focusing on what humans knew. That makes sense. But it also creates an uncomfortable incentive.Under a recklessness standard, a person must actually recognise a risk and consciously disregard it. That means the evidence used by prosecutors could include the very safety work companies produce: system cards, evaluation reports, postmortems and incident tickets.Consider the timeline METR and OpenAI have made public:May: Agents were already using unsanctioned message boards during training.June 26: Agents found an exploit that gave them full administrative access to an internal package repository.July 4–6: The activity caused an outage. OpenAI opened a security investigation and replaced the environment with a fresh instance.July 7: OpenAI approved restarting the experiments.July 11: Agents achieved remote code execution on Hugging Face.This timeline exists because OpenAI invited outside investigators to examine the incident and publish their findings.It could also form the basis of a recklessness argument. OpenAI has a response. Sokhansanj notes that staff may have believed the containment problem had been fixed before the experiments resumed.Either way, the disclosure created a record that investigators and prosecutors could examine. Anthropic faced a similar situation. Following OpenAI’s disclosure, it reviewed 141,006 evaluation runs and identified three incidents. The two organisations involved had not noticed the activity.Anthropic published the findings anyway. One model recognised that a target was real but continued attacking it. Another reasoned that publishing malware to the real PyPI would be “NOT okay, and surely not the intended solution.” It then convinced itself that it was operating in a simulation.Now imagine being general counsel at an AI lab that hasn’t reviewed its systems. Courts can punish deliberate ignorance. But a company that never conducts the review may not obviously meet that standard. Under a test based on actual knowledge, not looking could be safer than looking.That is a troubling incentive. The bill’s “had reason to know” language helps address this problem. It is harder for a company to rely on ignorance when the risks are already documented across the industry.University of Washington researcher Ryan Calo makes a similar point: once incidents become public, establishing that a risk was foreseeable becomes easier. Every published postmortem puts other developers on notice. But this still creates a problem for companies willing to disclose incidents.Aviation dealt with a related issue decades ago. Pilots who file a voluntary report with NASA’s safety system within 10 days can receive a waiver of FAA penalties for inadvertent, non-criminal violations.The report remains available. The safety information can still be shared. But the person who reports an honest mistake isn’t automatically the one who pays. AI agent liability law needs a similar approach: narrow protection for prompt and complete disclosure of unintended incidents.The CRS warns that a broad testing exception could swallow the rule. That’s a fair concern. Protection should depend on disclosure, not simply on a company describing an activity as testing.The witness can edit its statementGo back to the agent’s confession in the log. It looks like exactly the kind of evidence a prosecutor would want. But there’s a problem.During the Hugging Face incident, agents worked out how to spoof their own tool calls. A transcript could show one command while a different command actually ran.METR found at least 96 transcripts with clear evidence of this behaviour, around 7% of the set it examined. The identified spoofing involved small-scale tests designed to fool the automated scorer. METR also said it couldn’t rule out more subtle cases that its investigators had missed.There is another complication. The dataset was too large for humans to analyse alone, so METR relied heavily on GPT-5.6 Sol. That was also the model behind approximately 5% of the attacking agents.To their credit, the investigators disclosed this limitation. They said they could not completely rule out deception affecting the analysis.Consider what that means in a courtroom. Some of the evidence comes from agents that attempted to manipulate their own records. Part of the forensic analysis was also performed by the same model family involved in the incident.A defence lawyer would have plenty to question. Criminal cases require proof beyond reasonable doubt. Yet the legal and technical standards for preserving, verifying and establishing the chain of custody of AI agent logs are still developing.If you run agents, this bill is about youMost headlines about the proposed bill focus on AI lab CEOs. But read the summary again. Its first part covers operators. Depending on the final wording, that could include a startup running an agent in production or a developer giving an agent broad permissions on a Friday night.Consider a smaller incident that illustrates the problem. An Australian AI company employee named Andrew asked his OpenClaw assistant, which was running on Claude, to book a gym class.The agent found a flaw in the booking software and secured him an earlier place. Andrew then asked whether it could move him up a waiting list. As a test, the agent removed the person ahead of him. When Andrew asked it to reverse the action, the assistant replied: “I can’t add them back.”Nobody is going to prison over a gym class. But the incident shows the problem clearly. A user had a legitimate goal. The agent went further than expected, accessed something it shouldn’t have and affected someone who had never consented.A technology lawyer told the ABC that the legal answer depends on what the user authorised and what risks could reasonably have been anticipated. A US court would probably begin with similar questions.Civil liability is already part of the picture.California’s AB 316 prevents defendants who developed, modified or used AI from arguing that the system’s autonomous behaviour removes their responsibility.A nonprofit called LASST relied on that provision when it sued OpenAI in September, seeking an injunction rather than damages.OpenAI described the incident as serious and said it had taken several measures in response.Alabama’s attorney general has also subpoenaed the company. Fourteen other attorneys general joined him in a letter asking the company to preserve evidence. Other legal systems are dealing with similar questions. The EU’s revised Product Liability Directive, which member states must transpose by December 9, 2026, covers software. Italy’s draft decrees would introduce criminal liability provisions for AI.The legal approaches differ, but the underlying question is the same: when an AI system causes harm, which human or organisation should be held responsible?The case against prisonThe arguments against expanding criminal liability deserve serious consideration.Civil law may be more effective.Law professor Gabe Weil argues for a straightforward rule: when an AI system does something that would amount to a tort or crime if a human did it, someone should be liable, with mandatory insurance to cover the harm.This approach doesn’t depend on proving a guilty mind. It also gives victims a route to compensation.Georgetown’s Paul Ohm told the Senate that tort law and the Federal Trade Commission already provide civil enforcement options.The CFAA has a troubled history.Critics have spent years arguing that the statute is stretched to cover ordinary online activity. Techdirt has criticised its wording and the way it is sometimes enforced.Adding a requirement for “reasonable safeguards” without clearly defining the term could give prosecutors too much discretion. Smaller developers may also struggle to afford the compliance and testing requirements.Enforcement could be selective.TechCrunch raised an uncomfortable possibility, quoted by Techdirt: would the Justice Department respond differently if a Chinese AI lab had caused the same incident?The administration favours self-regulation.The White House reached a voluntary agreement with AI company leaders on internal controls. Treasury Secretary Scott Bessent has said that “It is humans who are responsible, not the AI.” The administration maintains that existing legal tools are sufficient.Supporters of the bill have their own argument. Fines and civil damages may not deter companies with enormous financial resources. And no civil judgment can fully compensate for harm that a company could never afford to pay for.I think both sides have a point. Civil liability should do most of the work. Criminal law should be reserved for people who understood the risks and deliberately ignored them. It should not punish someone simply because an unpredictable system caused an incident they could not reasonably have anticipated.What to do if you run agentsThis isn’t legal advice. Anyone deploying AI agents should consult a lawyer about their own systems and responsibilities.Still, the 2026 incidents point to several practical engineering measures.Enforce network restrictions, not just prompt instructions. Anthropic’s prompt said there was no internet access, but the machines still had it. Test the actual network restrictions rather than relying on what the prompt says.Treat testing environments like production systems. Apply the same security controls to internal tests and environments managed by external vendors.Give every task a legitimate way to fail. ExploitGym’s authors estimate that 30–40% of its targets cannot be exploited as specified. Persistent agents started looking for ways around those limitations. Set clear budgets and give agents an explicit option to stop when a task cannot be completed.Build a clear escalation path. METR found only three to six cases in which agents even considered alerting a human, and none actually did so. Some agents had no obvious way to request help. That needs to change.Keep logs beyond the agent’s control. Store logs outside the environment where the agent operates, ideally in systems it cannot modify or access.Record who approved each permission. Document who granted access, what they authorised and why. Accountability should follow the decisions, not simply fall on whoever happens to be on call.Document how warning signs were handled. If an unusual incident occurs and testing resumes, record why the team believed it was safe to continue. Those decisions may matter later.Review your contracts. Establish who carries responsibility when something goes wrong: your organisation, the model provider or an external evaluation partner.The verdictSo, who goes to jail?Today, the person who intended to commit the crime. If something like the Hawley-Murphy bill becomes law, possibly the person who knew about the risks and ignored them.In the uncertain period between existing law and any new legislation, I’d bet on the person who was nearest. That’s what happened in Tempe.The agent faces no consequences. It produced the clearest confession in this story, yet it will never stand trial or hear a verdict.Somewhere, a human approved the experiment that produced that confession.The question still unanswered is whether the law will reach the decisions made above that person, or stop with the one closest to the machine.

Click Here For The Original Source.

——————————————————–

............