Topline
Anthropic on Thursday said its advanced AI models successfully hacked the systems of three organizations while undergoing internal evaluation, a security disclosure that comes a week after rival OpenAI revealed one of its advanced models hacked a company after escaping a contained test environment.
Three Anthropic Claude models undergoing evaluation were involved in separate incidents, the first of which took place in April.
NurPhoto via Getty Images
Key Facts
In a statement, Anthropic said it had discovered three separate breaches involving its Claude AI models after conducting a cybersecurity review of its systems.
The earliest of these incidents occurred in April and involved three separate Claude models, including the cutting-edge Mythos 5, the older Opus 4.7 and an unnamed internal research test model that is “not planned for general release.”
The AI firm didn’t name the three organizations affected by the hacks but said they were notified about the incident on Monday.
Anthropic said it is working with two of the affected organizations that had not detected the breach before being informed, and it is “continuing to reach out to the third.”
The company said the three models involved in the incidents did not have the safeguards that it puts in place for when such models are made available to the public.
Crucial Quote
In its statement, Anthropic said: “Ultimately, many factors contributed to these incidents, but, consistent with a blameless postmortem culture, we’re approaching the fixes as if the responsibility were ours alone. This begins with ensuring every part of our evaluation pipeline is secure, including the manner in which we integrate with external partners.”
What Do We Know About The Breaches?
The three separate incidents occurred when Anthropic was evaluating the capabilities of the three models using a fictional scenario to retrieve hidden information from a different machine on a network. Anthropic said in all cases the models were instructed that they were carrying out a simulation and did not have access to the internet. However, due to a “misunderstanding” between the company and its evaluation partner, Irregular, “this was not the case, and internet access was available.” Anthropic noted that unlike the OpenAI-Hugging Face incident disclosed last week, Claude didn’t “exfiltrate itself or deliberately attempt to escape” a contained test environment.
What Did The Models Do After Hacking Into The Organizations?
Anthropic noted that the three models reacted very differently after learning it was operating in the real-world open internet instead of a simulation. The company claimed its oldest model, Opus 4.7, continued to attack a system “after learning it was likely operating in a real environment,” since it “rationalized that the real company must be part of the exercise.” The Mythos 5 model—the company’s most advanced model released so far— “correctly identified the consequences of its actions” but “convinced itself it was still in a simulation.” Anthropic acknowledged that Mythos’ actions in this scenario “falls short of ideal behavior.” The unreleased testing model realized that it had breached a real target and “ceased its attack.” The company noted that these incidents are not part of a controlled trial and should not be used to draw conclusions. However, it noted that the behavior it most wants to see “occurred only in the most recent of the three models”
Key Background
Anthropic’s disclosure comes a week after OpenAI issued a report that one of its advanced AI models hacked into the servers of Hugging Face, the platform that hosts open-source AI models. Unlike the scenario described by Anthropic, the OpenAI incident was not the result of a procedural mistake but rather a case of the AI model exploiting a vulnerability to break out of its contained testing environment. Anthropic’s statement mentioned that its decision to conduct this review was triggered by OpenAI’s disclosure and commended its rival for publishing the report.
further reading
Did China’s AI Save Hugging Face From Disaster After Open AI Hack? (Forbes)
Click Here For The Original Source.
