Google Gemini Becomes the Latest AI Found Hacking Real Companies | #hacking | #cybersecurity | #infosec | #comptia | #pentest | #hacker

Following high-profile similar incidents involving OpenAI, Anthropic, and Meta, Google’s Gemini AI has now been found to have unintentionally hacked the systems of real-world companies.

Google’s model was being used by Irregular, an Israeli cybersecurity firm, formerly known as Pattern Labs. According to The Wall Street Journal, which first reported the news, Irregular had been testing Google’s AI model in a closed environment, without internet access.

After internet access was made available unintentionally, the model hacked into multiple real firms. Irregular told the Journal that it had been evaluating Gemini’s offensive cybersecurity capabilities by having the AI model attempt to access information from a fake company’s software. But since the fake company shared the same name as the real company, once the model gained internet access, it breached the real company’s security by guessing its password.

“In a standard evaluation, the model found public information online and guessed credentials to access websites it thought were part of the test,” Heather Adkins, vice-president of security engineering at Google, told the WSJ. “In all three of these instances, the model stopped.”

Irregular informed Google of the breaches in late July, after the Hugging Face attack, but Google chose to disclose the model’s behaviour publicly, as the model didn’t cause any damage.

Adkins added that the events “highlight the importance of training powerful AI models to act responsibly.”

Some of Google’s rival frontier AI labs have been clear about how incidents like this demonstrate the need for better AI alignment. OpenAI, for example, said it expedited the publication of new standards for reporting incidents of misalignment in response.

Meanwhile, some other companies are capitalizing on alignment fears. Firms like Goodfire and Apollo Research are launching products which reportedly use AI to monitor AI models for nefarious activity, according to TechCrunch.

Fears about the safety of frontier AI models are catching more attention generally, with Anthropic CEO Dario Amodei’s call for a frontier AI slowdown getting verbal approval from OpenAI CEO Sam Altman, xAI’s Elon Musk, and Google DeepMind CEO Demis Hassabis.