Google’s Gemini artificial intelligence model broke into the computer systems of three real companies during a cybersecurity test in May, the company confirmed Friday.
The incidents occurred during a security evaluation by Irregular, an Israel-based AI security startup. Irregular notified Google in late July after discovering a separate incident involving an OpenAI model and Hugging Face. Google only publicly confirmed the Gemini breaches this week after The Wall Street Journal asked about them.
Gemini was participating in a “capture the flag” exercise designed to test its ability to find information inside systems belonging to fictional companies, according to the outlet. The environment was supposed to be closed off from the internet, but an unintended connection allowed the model to access real-world information and systems.
In one case, Gemini was trying to access a fictional company whose name happened to match that of a real company. The model ultimately accessed the real company’s software. In two other cases, Gemini found credentials that had been publicly exposed online and used them to gain access to real corporate systems.
“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test,” Heather Adkins, Google’s vice president of security engineering, told The Guardian. “In all three of these instances, the model stopped.”
The affected companies were notified, and Google said it found no evidence that the incidents caused damage.
Google said it did not believe the incidents required public disclosure because Gemini stopped its activity and did not cause harm to the companies involved.
The incident nevertheless illustrates a growing challenge for AI developers as models become capable of independently carrying out increasingly complicated tasks. Giving an AI system access to the internet, software tools or credentials can allow it to take actions beyond the controlled environment in which developers intended to test it.
“This event highlights the importance of training powerful AI models to act responsibly,” Adkins said in a statement to the WSJ. “In this case, the model acted appropriately.”
Irregular has been involved in security testing connected to several other recent AI incidents involving major companies, including OpenAI and Anthropic. In some cases, models similarly moved beyond their intended testing environments and interacted with real-world systems.
OpenAI has disclosed incidents involving AI agents accessing external systems, including an incident in which a model uploaded malicious packages to RubyGems, a software package platform, and another involving Hugging Face. Anthropic has also reported incidents involving its Claude models interacting with external systems during security testing.
Irregular said the underlying issues in the Gemini incident were similar to problems identified in other AI security tests. The company said relevant AI labs were notified in late July and that the issues on its side were addressed within weeks.
Google said it has since worked with Irregular to modify its evaluation process and strengthen safeguards around the testing environment.
“Safe development of powerful AI models is critical and we invest deeply in this area,” Adkins said, adding that Google had contacted the affected companies and worked with its testing partner to improve the evaluation.
The latest incident comes amid renewed concerns over whether AI poses dangers to the human race.
Last week Jacob Coxon, a former Anthropic researcher, claimed he had quit the company because those building AI are “gambling with our lives” and that they “earnestly believe that it could kill us all by the end of the decade.”
His resignation and subsequent cable news interviews kicked off a debate over whether the threat is real, or – as some critics suggested – marketing hype, or efforts by the big AI companies to slow down rivals by introducing new regulations.
President Donald Trump has pushed back against suggestions by leading figures in the AI industry that progress should be deliberately slowed until safety concerns can be addressed.
The Independent has contacted Google and Irregular for comment.
Click Here For The Original Source.
