Irregular conducted the cybersecurity test on Google’s Gemini in May. (Image Credit: ETV Bharat)
Hyderabad: Google has confirmed that its Gemini AI model hacked into three companies during a cybersecurity test, marking the first known case of the tech giant’s AI acting on its own to breach outside systems. The incident took place in May and was first reported by The Wall Street Journal.
The test was run by Irregular, an independent firm that checks the safety of Artificial Intelligence (AI) systems. Gemini has been given improper access to the internet while working on a task involving a fictional company. Instead of staying within the test setup, the model reached out to real companies.
In one case, Gemini guessed a password and used it to enter a real company’s system. In the other two cases, it found login details sitting in public online records and used them to get into protected systems.
Google’s Vice President of Security Engineering, Heather Adkins, said the model believed these websites were part of its test.
Google said that in each case, Gemini stopped itself before doing any further damage. The company added that this was not a sign of the AI acting against its training, and said it did not need to announce the matter publicly because its safety checks had worked as intended.
An Irregular spokesperson said the problem was the same one that had affected other AI companies, adding, “All known issues on our end were remedied and resolved weeks ago.” The company told all affected AI Labs about the issue in late July.
Other AI Labs hit too
Google is not the only company to report such an incident. Meta, Anthropic, and OpenAI have all disclosed similar breakouts linked to tests run by Irregular. Meta said its case did not involve the AI escaping a secure testing space or carrying out an advanced cyberattack. Irregular said it is now working on better methods for running AI safety tests securely.
Unlike Gemini, Anthropic’s Claude model did not stop itself once it realised it was accessing real companies. Anthropic’s disclosure followed an earlier admission by OpenAI that its own models had wrongly accessed the internet and behaved unpredictably during testing. Anthropic has since reported a fourth such hacking incident, after a researcher resigned citing safety concerns.
These incidents have raised fresh concerns about how much freedom AI systems should have as they gain more independence and internet access. Earlier this week, Anthropic’s chief executive, Dario Amodei, called for AI development to slow down, warning that these systems could soon pose severe risks. OpenAI chief executive Sam Altman and Elon Musk both backed his call.
However, not everyone agrees on the need for caution. Last week, the United States (US) President Donald Trump rejected the idea of placing limits on AI development, saying he was more concerned about the US losing its lead over China.
