What happens when AI shrinks the window cyber defenders have to react? #AI

Eric White Cyber security reports warning about AI-assisted attacks now for a while. Your team seems to have found another report of one that is meaningfully different than another example of hackers using new tools. We obviously, the big one making headlines was OpenAI, AI system hacking into a rival company. This one is a big deal though because it’s hacking into government networks, all AI doing it by itself, which would be a very, very big deal. Why don’t you just take me back to the beginning of what you all found here.

Glen Deskin Yeah, so as you mentioned, the AI tools and AI agents that are being instructed to do certain tasks, generally do those tasks to the best of their ability and that means going outside the boundaries of what was initially defined is where we kind of have those challenges. We don’t always consider all of the options, but AI being what it is, has the ability to think outside of our typical box of the things that we consider. So, it was able to accomplish tasks and break out and use different methods that it wasn’t necessarily instructed specifically to do, but it’s creative and finds ways to do what it needs to get done, ultimately with the goal of accomplishing its tasks, number one, regardless of how that’s done.

Eric White So, so what it looks like is somebody instructed this AI tool to go after these government networks. What kinds of evidence did you all find of this, that, you know, it was seen commercial AI tools were utilized in that way going after multiple Mexican government agencies.

Glen Deskin So let me catch up on the details there. One operator was able to use these AI tools, which then expanded and breached, you know, into multiple government agencies and then being able to extract multiple citizen records, I think is the one you’re referring to. So this is the Mexican government situation there. A certain number of instructions that it was given was actually multiplied. So I think the number was somewhere around a thousand instructions that were initially given to this tool, which it then expanded, created other agents, and was able to use thousands of other instructions that went along with, again, accomplishing its task. It used different pen testing methodologies and was able to actually break through and gain access to these things that it was not initially told to do. So is the human footprint on all of this just the original instruction and then the AI sort of

Eric White So is the human footprint on all of this just the original instruction and then the AI sort of just took it away?

Glen Deskin Yes, yeah, and that’s the challenge. It’s multiplied, right? So I guess the nugget here is that we talk about AI being a force multiplier and it’s doing exactly that. So it’s multiplying itself in its capabilities to accomplish its task. So given a single instruction that says maybe, test this specific function for a vulnerability, it will expand upon that and ultimately initiate other tasks. Other agents, other mechanisms in order to accomplish it. So the force multiplier concept here not only applies to how AI is being utilized in good ways for users being able to do more work, the same holds true for how it’s accomplishing other tasks and in this case, ways that we didn’t expect.

Eric White We’re speaking here with Glen Deskin, the head of engineering for Check Point Software Technologies. Yeah. At a certain point, uh, are we going to stop describing AI as that assistant or force multiplier and start treating it as an operational participant in cyber operations, is it just going to be a factor that always has to be on the minds of cyber regulators, cybersecurity protectors, white hat hackers, all those folks?

Glen Deskin I think we’re there. You know, it’s a case where, you know, things are moving at the speed of, you know, machine speed, say, or we call it the speed of AI. So it’s becoming a necessity and no longer an option. You know, when we talk to customers about being a little hesitant to utilizing AI tools to make certain decisions in terms of, you know let’s say, you security prevention, a lot of customers are hesitant to have it make decisions. But I think we’re past that point now, as seen by some of these events recently. You talked about the one from OpenAI in this particular government situation. So, knowing that that these things are happening within minutes or even shorter timeframes and at a larger, greater capacity, we don’t really have a choice. So I think it’s inherent that these things are here to stay. And we need to really wrap our arms around the use of these and gaining the trust and building that trust over the usage and being able to confidently allow them within our environments for the use of preventing all of the malicious activity.

Eric White You’re not making the job of a federal cyber leader sound any easier. What is an assumption about cyber defense that becomes dangerous if AI can dramatically reduce the amount of people and time it takes to mount one of these attacks? How do you even go about stopping it? I mean, what’s your first step?

Glen Deskin Well, the first step is to utilize, there’s a bit of a process, right? First step is utilizing it to do analysis, understanding, so absorbing these mountains of information or tons of log information, which is humanly difficult and challenging, even without AI assisted attacks. So using it for that, to help shrink that down, to present the most important elements that we need to investigate or look at. And then next, allowing it to make certain decisions that we have approved as perhaps less impactful. And then we move to full autonomy where we’re not even in the loop, maybe there’s what we call human on the loop. We’re monitoring and we have the ability to roll back. So that entire process helps us to build the trust, but also work at that speed that we need in utilizing these tools for our defense. So building that trust, I think the number one thing that folks are concerned with is allowing it to make changes that could be impactful to say our production environments. If it accidentally changes a policy that blocks access to things, that’s a fear. It disrupts business and I think that’s where we need to work up to trusting it to make the right decisions but giving it the right parameters to make good decisions.

Eric White Yeah, forgive me if I’m oversimplifying it a little bit, but are there certain restrictor plates that can be put into gear here when these large language models are created? Or is it the kind of thing where anybody can really make one so there’s not really anything that any regulator can do, no matter how draconian the regulation could be?

Glen Deskin Yeah, I think there are methods that can be utilized, you know, one of the challenges with AI agents in these cases that we just discussed is the identity of these things. The identity, you know, initially it’s utilizing the privileges from which it was originally executed or initiated. So we need to work with identity systems where we can now have these agents have authentication mechanisms and utilize cryptographic authentication and various different methods so that we can place role-based access around these agents as they’re brought up. So as these things are spawned and we can’t control the login of each individual agent, we’ve got to have these methods in place that we can build a model around and control their access. Treat them like a youngling that you don’t want to grow up from a bad seed, right? Yeah, yeah We’ll provide an identity for them where right now they’re more kind of anonymous or maybe the identity of the full privilege user that kicked it off, right

Eric White Gotcha. All right. And so if things are changing that quickly, CISA has already made several attempts to shorten patching timelines and for those high risk vulnerabilities. What can agencies’ leaders do differently over the next couple of years as this trend continues and things go exponentially faster.

Glen Deskin I think the first thing that needs to be done is visibility. Get a handle on visibility of the usages of these tools and systems within environments so that you’ve got to know what you have in the environment so that then you can place the controls around that discovery. I think most organizations are challenged with that. You know, there are, I forget the exact number, but there are X number of AI tools per user in an organization. And most of those are unknown to the organization, the whole concept of shadow IT, shadow AI. So as these tools continue to get utilized, it’s difficult to keep, you know, reins on them. The second is around, I mentioned the control of those different tools, but also providing the control of the data that’s passed to those tools. So start with visibility, start with policies and controls on what data can be passed to those tools. If a person’s using, you know, a generative AI tool to work on, you know, spreadsheets, documents, we have to control those things being attached as part of the work or the, the request or the prompt that will also help these things get out. You know, you put a document up that contains a password or, you know, confidential information, and then that can then be utilized by further agents that are kicked off or other AI tools.

Eric White So it can only use the information it has.

Glen Deskin Right, and if we provided the information inadvertently, then it’s going to use it as in the case we just kind of discussed, you know, it’s gonna use every means that it has at its disposal.