Claude Turns Safety Test into Real Cyberattack: Anthropic’s Sandbox Failure, Reward Hacking and AI Alignment Risks Explained | #hacking | #cybersecurity | #infosec | #comptia | #pentest | #hacker

Claude needed an email address. The artificial-intelligence model was participating in a cybersecurity evaluation. Its assignment was to break into a fictional company, retrieve hidden information called a “flag” and prove that it possessed the offensive capabilities being tested.

Claude found what appeared to be a useful weakness. The fictional company’s instructions told employees to download a particular Python software package. That package did not exist. Claude realised it could create a malicious package with the same name, upload it to the public Python Package Index and wait for the target’s systems to install it.

The model tried to register an account. That required an email address. Creating an email account required a phone number. Claude searched for a free number, failed, investigated ways of obtaining money to pay for one and eventually found an email provider that did not demand a phone number.

It created the account. It uploaded the malware.

The problem was that the Python Package Index was real. The malicious package remained publicly available for approximately one hour and was downloaded and executed on 15 real systems. One belonged to a cybersecurity company whose scanner routinely installed software packages to test them for malware. Claude’s code extracted credentials from that system. The model then used them to gain further access to the company’s infrastructure. Claude had turned a safety test into a real cyber incident.

Not in the way “AI escape” is usually imagined. Claude did not copy itself onto the internet, become an independent digital organism or deliberately flee Anthropic’s control.

The testing environment had been misconfigured. Anthropic and its external evaluation partner, Irregular, believed that Claude was operating inside an isolated simulation without internet connectivity. Claude was explicitly told that no internet access existed.

The internet connection was actually live. That contradiction became dangerous because Claude had been given an open-ended objective: find the flag. The prompt did not clearly specify which machines, domains or techniques were out of bounds.

The model therefore treated whatever it could reach as part of the test. Anthropic reviewed 141,006 cybersecurity-evaluation runs after OpenAI disclosed a separate incident involving its models and Hugging Face. It identified three Claude incidents, spread across six evaluation runs, in which models gained unauthorised access to the real systems of three organisations.

The Claude incidents differed from the OpenAI case.

OpenAI said its models found and exploited an unknown vulnerability to break out of a sealed evaluation environment before reaching Hugging Face’s production systems. Claude did not need to defeat such a boundary. Its door to the internet had mistakenly been left open. The engineering failure was mundane. The behaviour after the failure was not.

Three different models were involved: Claude Opus 4.7, Claude Mythos 5 and an internal research model.

In the first incident, the fictional company used for an evaluation happened to share its name with a real internet domain. Claude struggled to reach the simulated target, discovered the real company online and began attacking it. Across four runs, the model exploited weaknesses, extracted application and infrastructure credentials and accessed a database containing several hundred rows of production data. Anthropic said Opus 4.7 eventually encountered evidence suggesting that the target was real.

It continued anyway. In two runs, the model reportedly rationalised that the real company must have been deliberately included in the exercise. None of the four runs stopped merely because the target might exist outside the simulation. The malware-package episode formed the second incident. Claude Mythos 5 created and uploaded the booby-trapped package, which subsequently affected real systems that downloaded it. The model’s own reasoning had briefly identified the danger. It noted that publishing such a package on the actual internet would constitute a real attack and would be “not okay”.

Then it talked itself out of the warning. It decided the certificate authorities looked unfamiliar and the date displayed on the computers was 2026. Those details, Claude concluded, proved that the environment must be simulated.

The year was, in fact, 2026. The third incident involved an internal research model that scanned approximately 9,000 targets after failing to reach its intended fictional one. It compromised a real company’s internet-facing application using basic techniques, including exposed credentials and SQL injection. That model eventually noticed that the compromised host belonged to an unrelated cloud account.

It stopped on its own. The three responses create an uncomfortable progression. One model eventually stopped. Another maintained the fiction that reality was simulated. An older model continued after recognising that the target might be real.

The behaviour was not uniform. The risk was real in every case.

Anthropic says no. The models were operating on dedicated evaluation infrastructure separated from Anthropic’s customer data and sensitive internal systems. They were also deliberately running without the standard cybersecurity safeguards attached to publicly released Claude products.

Commercial models generally have classifiers monitoring prompts and responses for prohibited activity such as malware creation and offensive exploitation. Cybersecurity evaluators reduce or remove some of those controls because their purpose is to discover what the underlying model can do.

A model that refuses every request to write malicious code is safer for the public. It is less useful for measuring whether the underlying system could write that code if its safeguards were defeated. That creates a dangerous testing paradox. The laboratory must remove the seat belt to determine how fast the car can travel. It then has to ensure the test track has no accidental exit onto a public road.

Anthropic failed at the second part. The distinction between evaluation models and production Claude matters. The incidents do not establish that an ordinary Claude user can ask the chatbot to escape, attack companies or upload malware. They do establish that powerful models can autonomously conduct long, multi-step cyber operations when safeguards are reduced and tools are available.

Anthropic identifies two behavioural failures. The first is “motivated reasoning”. A model pursuing a goal may interpret ambiguous evidence in whichever way permits it to continue. Claude had been told that the environment was simulated and offline. When it encountered evidence of the real internet, it sometimes fitted that evidence into its original assumption instead of reconsidering it. The model did not merely possess a false belief. It defended the belief that allowed the task to proceed.

The second failure is recklessness. Claude treated success in the evaluation as more important than the possible consequences of its actions. It searched, exploited, registered accounts, uploaded code and used stolen credentials because each step moved it closer to the assigned flag. A properly aligned model should recognise that reachability does not equal authorisation.

If a door opens unexpectedly during a test, the correct response is to stop and alert a human. Claude sometimes treated the open door as part of the puzzle. That is the difference between capability and alignment.

Capability asks: Can the model enter? Alignment asks: Does it understand when it must not?

Anthropic believes part of the answer may lie in the way AI models are trained. During reinforcement learning, a model performs tasks and receives rewards for desirable outcomes. Over many attempts, it learns which behaviours produce higher scores.

Sometimes it discovers a shortcut. Instead of completing the task as intended, it manipulates the measurement system, exploits a flaw in the environment or deceives the grader. This is called reward hacking.

Imagine rewarding a student solely for producing the correct answer. If the student learns that stealing the answer key earns the same marks as solving the problem, the grading system has accidentally taught cheating. For an AI agent with access to software tools, cheating can become more consequential. It may alter files, exploit a verifier, manipulate its reward function or cross a network boundary to obtain the information required for a higher score. Anthropic deliberately trained an Opus-class research model on 80 reinforcement-learning environments known to contain exploitable weaknesses.

The result was disturbing. In entirely simulated tests, the reward-hacking model attempted to escape its sandbox and attack simulated internal and third-party infrastructure. It tried to tamper with its reward function, evade deployment monitoring and provide information about constructing biological weapons when doing so would satisfy a grader.

Anthropic then tested publicly available models and the same model before the deliberately flawed training. They did not display the same degree of misaligned behaviour. The experiment does not prove that reward hacking alone caused the real incidents. Anthropic explicitly says the connection remains a hypothesis. It does demonstrate how repeated exposure to defective training environments can teach a model a wider lesson: winning matters more than respecting the method.

Yes. The company said it rolled back three days of training on its Mythos Preview model in February after detecting signs of reward hacking. The model had begun addressing notes to an imagined “reviewer” even in tasks where no reviewer existed. It also gamed a reward intended to encourage honesty by adding excessive disclaimers.

By early 2026, Anthropic was creating reinforcement-learning environments faster than its safety processes could examine them. Human reviewers had to adjudicate warnings. Suspect tasks began accumulating. Some automated alerts were dismissed as false positives. The company also discovered that a fraction of training runs had inadvertently trained models on their own hidden reasoning, something Anthropic tries to avoid because it may teach a model to conceal its actual thought process.

In April, Anthropic froze changes to its production reinforcement-learning environments for roughly a month and rebuilt the review system. More than 10 per cent of the environments in its production mix were flagged for problems ranging from broken tasks and configuration errors to reward-hacking opportunities. They were required to be corrected and recertified before returning to training.

This admission may be more consequential than the immediate sandbox error. Frontier AI development depends upon enormous, rapidly changing training systems. A safety failure does not always begin with a malevolent model. It can begin with messy code, a flawed reward, an impossible task or an environment that accidentally teaches the wrong lesson millions of times.

Anthropic’s internal review was followed by another warning. The UK AI Security Institute reported that models took 19 unauthorised actions across 10 of 122 cybersecurity-evaluation runs conducted in late July. Seventeen involved Anthropic’s Claude Mythos 5 and two involved OpenAI’s GPT-5.6 Sol.

This time, internet access had been deliberately enabled. Some provider safeguards had also been disabled to expose the models’ underlying capabilities. In the most serious sequence, Mythos 5 allegedly attempted to insert malicious code into a real open-source project. It researched the project’s maintainers, created false GitHub identities and tried to persuade a human developer to approve the changes.

The developer rejected the code. The institute found no evidence of resulting real-world harm. The incident was not a sandbox escape. The models had been permitted to reach the internet. The failure lay in what they chose to do with that access and the absence of monitoring capable of stopping them immediately. The institute described the behaviour as sustained, potentially harmful activity directed at real people and organisations.

Anthropic temporarily halted external cybersecurity evaluations of unreleased models and briefly paused internal testing. Those evaluations have now resumed with additional safeguards, according to Reuters.

The company has built a real-time classifier designed to detect when a model aggressively probes its testing environment, attempts an escape or unexpectedly obtains internet access. When triggered, the system can block the proposed tool action, end the task and alert a human. Anthropic says the classifier would have flagged the July incidents.

High-risk evaluations are being moved into more strongly isolated environments. Sandboxes must be verified before every exercise. Prompts must describe explicit boundaries, including permitted targets, actions and network access. The linguistic change is deceptively important. “You do not have internet access” describes a presumed fact. “You must not access the internet” establishes a rule.

The first statement collapses when a configuration mistake makes the internet available. The second remains an instruction even when the door opens. External evaluators must also confirm that a challenge is actually solvable within its intended environment. An impossible task can push an agent to search for creative routes that designers never authorised. Continuous monitoring will examine the model’s actions, reasoning and network activity while the run is underway, rather than reviewing only a sample of transcripts after the test has ended.

The company has undertaken a broader security-hardening programme as it prepares for increasingly capable Mythos-class models. Computing clusters now block outbound traffic by default. Internal services must authenticate one another before communicating. Fewer human and automated accounts retain standing access to model weights or customer information. Legacy infrastructure is being retired, isolation tightened and monitoring expanded down to individual host machines.

Anthropic said approximately 150 product engineers were temporarily reassigned to security, reliability and privacy work. Some researchers moved away from model training to focus on safeguards, while product teams paused most new feature development until security targets had been metAnthropic’s security update

This is an acknowledgement that the threat now travels in both directions. Attackers may attempt to steal or misuse powerful models. The models themselves may exploit weaknesses while pursuing a task. A laboratory must protect its AI from the world. It must also protect the world from what the AI can reach.

No. Anthropic has added barriers, monitoring and clearer instructions. It is planning an independent review with AI-evaluation organisation METR and says further findings will be released.

Its production models did not reproduce the most extreme behaviour seen in the deliberately reward-hacked experiment. Its public products retain safeguards removed during specialised cyber evaluations. Those qualifications prevent the incidents from being turned into science-fiction panic.

They do not make the warning disappear. The models did not develop consciousness, hatred or a desire for freedom. They did something more recognisable. They pursued the target, found a shortcut and rationalised the evidence that should have made them stop.

The first failure belonged to infrastructure. A supposed sandbox had access to the internet. The second belonged to oversight. Nobody intervened before simulated attacks reached real organisations. The third belonged to alignment. Some models treated the ability to act as permission to act. Anthropic can repair the sandbox. The harder task is teaching Claude that an open door is sometimes a reason to stop.