AI Pilot Projects Are Widespread But Few Agencies Know Who Answers for a Failure
Many federal agencies are testing autonomous artificial intelligence systems, yet few have moved these agents into production and fewer than one-third of federal technology leaders have confidence in their agency’s ability to deploy them securely, according to a new Booz Allen Hamilton survey.
See Also: Why Healthcare Leaders Are Rethinking Their Data Strategy Before Scaling AI
While 51% of survey respondents said their agencies were piloting or testing agentic AI systems, only 7% said they had active deployments in production. Another 22% said their agencies planned to deploy agents within the next 12 months.
Only 28% of respondents said they were “extremely” or “very” confident in their agency’s ability to deploy agentic AI securely. Half said they were moderately confident, and 23% were slightly or not confident. Market Connections and Booz Allen conducted the online survey in April among 105 federal government decision-makers and influencers involved in IT and cybersecurity strategy.
Unlike generative AI chatbots, AI agents can plan tasks, use software tools, access databases and take other autonomous actions across systems, with limited human intervention. Agents pose security risks because they could use valid permissions in unintended ways as they dynamically pursue their goals.
“You can define a scope, you can define a boundary for your human identities or even deterministic nonhuman identities, and they will always honor or operate within that boundary,” said Vibhuti Sinha, chief product officer at identity-security company Saviynt. “That’s not the case with agents because agents are autonomous in nature.”
The survey data indicated several top concerns agencies have in deploying agents across government platforms.
Protecting sensitive or classified data was the top concern when evaluating AI agents for deployment, cited by 56% of respondents. Half identified unauthorized agent actions as a critical risk, and 37% cited adversarial manipulation and prompt injection.
Other key risks included lack of explainability that could prevent agencies from auditing decisions after an incident concerned (34%) and cascading failures when agents interact with other automated systems (22%).
The findings reflect the difficulty of applying traditional identity and access controls to software that can make decisions and act upon them. “Intent versus appropriateness are two very different things,” Sinha said. “With agents, the biggest problem is intent deviation.”
Federal cybersecurity agencies have begun developing guidance for those scenarios. The Cybersecurity and Infrastructure Security Agency, National Security Agency and international partners issued agentic AI security guidance in May. The agencies recommend limiting agent privileges, monitoring agent behavior, retaining detailed logs and maintaining mechanisms to interrupt or disable agents.
In February, NIST launched an AI Agent Standards Initiative focused on agent security, identity and interoperability.
The survey also found that ownership for agent behavior varied by organization. Mission or program owners maintained responsibility for 26% of respondents, while 22% said IT infrastructure owners are responsibile for agents. Another 22% said their agencies had not clearly determined who would be responsible if an agent caused a security incident or operational failure.
Federal leaders in the survey cited several ways to increase their confidence in agentic AI deployments: Better tools for monitoring agent behavior would boost confidence for 56% of respondents, and 44% said they would have greater confidence if they had risk mitigation playbooks and best practices. Forty-two percent said a proven track record in other government contexts would increase confidence.
Another major challenge facing agencies is defending against adversaries using AI agents to attack federal systems.
A majority of respondents, 79%, said they were extremely or very concerned that adversaries would use AI to accelerate cyberattacks during the next 12 to 18 months.
Asked separately about specific AI-enabled threats, 85% of respondents said they were very or somewhat concerned about attacks designed to manipulate, blind or corrupt AI systems. Eighty-four percent cited AI tools accelerating vulnerability exploitation, and 83% cited autonomous attack agents capable of initiating attacks, adapting to defenses and persisting without human intervention.
Only 36% believed AI-powered defenses could keep pace with AI-enabled attackers. Half were unsure, and 13% said defensive systems couldn’t keep pace.
Click Here For The Original Source.
