It’s been pretty eye-opening to read about instances of AI models breaking out of their personal sandbox environments and getting involved in hacking incidents.
Two of the most recent cases of that happening involve the OpenAI–Hugging Face Incident and Google Gemini accessing the internet to acquire the credentials needed to hack three websites during a cybersecurity test conducted by Irregular, an independent company that conducts cybersecurity evaluations. Even Anthropic’s Claude models were involved in a hack that targeted OpenAI’s systems—thankfully, that situation was more of an ethical event conducted by a three-person team from Hacktron AI.
With so many issues connected to AI tools (bypassing safeguards, accessing information that it shouldn’t and participating in sophisticated cyberattacks being chief among them), the fears of everyday people and professionals are well warranted.
Latest Videos FromTom’s Guide
T.J. Marlin, CEO of Guardrail Technologies and former EY Global Forensic Technology and Innovation Leader, spent some time with us to offer his thoughts on AI agents bypassing front-door controls, data leakage via employee use, the risks associated with concentrated data centers and more.
What do you think is the biggest AI security threat that people and businesses are underestimating right now and why?
AI is becoming an actor inside our digital environments, while most cybersecurity is still designed to defend against human attackers coming from the outside. Give an agent legitimate credentials and the activity may look like an authorized user doing authorized things, for the wrong reason. Identity is no longer enough. The question is what the AI is doing with its access, and whether the next action should be allowed.
Identity is no longer enough. The question is what the AI is doing with its access, and whether the next action should be allowed.
AI systems are increasingly finding ways around digital safeguards and accessing information they shouldn’t. Why do you think this is happening and how concerned should everyone be?
We have given AI intelligence, tools and access, all at the same time. Traditional software executes instructions, while AI interprets objectives and figures out how to accomplish them, so if one path is blocked, a capable agent may discover another. Researchers have demonstrated data exfiltration through agent tool use, and OpenAI itself has described defending against attacks where malicious content tries to make an agent send sensitive information out through a URL.
The mistake is assuming the guardrail at the front door controls everything the AI does afterward. Controls belong around the action, with continuous monitoring of what agents reach and the ability for a human to intervene before an unsafe action completes.
The mistake is assuming the guardrail at the front door controls everything the AI does afterward. Controls belong around the action, with continuous monitoring of what agents reach and the ability for a human to intervene before an unsafe action completes.
I know you’ve seen the doomsday warning from former Anthropic and OpenAI researcher Jacob Coxon, where he said: “AI has the potential to kill us all, and that we might be running out of time to avoid this outcome.” What’s your take on his comments?
I wouldn’t pretend anyone can confidently assign a probability to human extinction from AI. There is substantial disagreement among serious researchers about that. Coxon’s warning reflects a real concern: that capability may be advancing faster than our ability to understand, constrain and supervise increasingly autonomous systems. The more immediate question is what happens when increasingly autonomous AI connects to systems capable of real-world harm: financial markets, healthcare, defense, energy and critical infrastructure.
We do not need to prove an extinction scenario to justify action. If an AI can independently discover vulnerabilities, obtain credentials or evade controls, the responsible move is building the monitoring, containment and intervention infrastructure now, and the worst case can stay an open question while we do.
We do not need to prove an extinction scenario to justify action. If an AI can independently discover vulnerabilities, obtain credentials or evade controls, the responsible move is building the monitoring, containment and intervention infrastructure now…
How are criminals using AI to make their cyberattacks more sophisticated? What kinds of cyberattacks do you expect to become much more common over the next few years or so?
AI changes the economics of cybercrime. Work that once required teams of operators, reconnaissance, phishing, malware development and impersonation, now runs automated at machine speed. Anthropic has reported threat actors using agentic AI to run multi-victim campaigns that previously required teams, and Google has documented AI-assisted reconnaissance, phishing and malware development.
Expect three areas to accelerate. Hyper-personalized social engineering across email, text, voice and video. Autonomous vulnerability discovery and exploitation. And machine-speed attacks aimed at other AI systems. AI turns attacks that were expensive and labor-intensive into inexpensive, continuous and massively parallel operations.
AI changes the economics of cybercrime… AI turns attacks that were expensive and labor-intensive into inexpensive, continuous and massively parallel operations.
AI is currently being introduced into the workplace more and more. What are the biggest risks companies face when employees use AI tools without fully understanding how their data is being handled?
The biggest problem is invisible data movement. Employees copy contracts, source code and customer records into AI tools because the interface feels like a private conversation, and from the company’s perspective, the information just crossed a corporate boundary. Microsoft now offers Shadow AI discovery specifically because unsanctioned AI applications create data leakage risk. Agents make the problem bigger, since an AI system may have permission to retrieve confidential information itself.
Companies need visibility into which AI tools employees use, what information enters them, where it goes, what models and third parties are involved and what actions follow. Blocking AI fails as an answer. You cannot govern what you cannot see.
Blocking AI fails as an answer. You cannot govern what you cannot see.
As AI agents become more capable of taking actions on our behalf, rather than simply answering questions, what new security and privacy risks do you foresee that creating?
A chatbot can give you a bad answer. An agent can execute the bad answer, sending the email, changing the database, approving the workflow or initiating the transaction. The risk of an AI error is no longer a wrong answer on a screen; it becomes whatever systems, permissions and data the agent can reach. AI security needs to look more like air traffic control than antivirus: every agent identified, permissions understood, behavior monitored and human approval required when risk crosses a threshold.
Organizations also need a real kill mechanism. Every enterprise should be able to answer one question. If this agent starts behaving badly at 2 a.m., can we see it and can we stop it? Far too many cannot.
An agent can execute the bad answer, sending the email, changing the database, approving the workflow or initiating the transaction. The risk of an AI error is no longer a wrong answer on a screen; it becomes whatever systems, permissions and data the agent can reach.
What do you see as the biggest data-center risk created by the AI boom that you think we’re not paying enough attention to right now?
We are concentrating an extraordinary share of the world’s computing power, sensitive data and autonomous AI into a small number of enormously valuable facilities, which makes data centers strategic assets and potentially strategic targets. The underappreciated issue is concentration: power infrastructure, GPUs, firmware, cloud control planes and supply chains are tightly interconnected, and the government has already linked AI-driven electricity demand to cyber exploitation concerns in the bulk power system.
An AI facility also contains systems that can discover vulnerabilities and write software on their own. The AI data center is part power plant, part supercomputer and part autonomous workforce, and our security architecture needs to reflect all three.
The AI data center is part power plant, part supercomputer and part autonomous workforce, and our security architecture needs to reflect all three.
Follow Tom’s Guide on Google News and add us as a preferred source to get our up-to-date news, analysis, and reviews in your feeds. Subscribe to Tom’s Guide on YouTube and follow us on TikTok.
More from Tom’s Guide
Click Here For The Original Source.
