The breaking news last week that OpenAI agents had hacked the Australian public health service and three other agencies while doing what was supposed to be a rather mundane piece of research is the latest example of AI escaping human control.
On Friday, it emerged that the same company’s agents – the name for its more autonomous AI tools that can work without continuous instruction – “meddled” with US government websites, including those of the department of education and the Securities and Exchange Commission, the regulator of the world’s largest markets.
“It feels like a moment,” says Jonathan Ruane, a senior lecturer and research scientist at Massachusetts Institute of Technology and artificial intelligence specialist, not just because of that news but because of the wider recent advances in the technology’s abilities. There was the remarkable Hugging Face hack on a hub for AI developers themselves, also involving OpenAI’s agents, and the growing awareness at highest echelons of politics and big business as to how fast all of this is moving.
Ruane draws the line at human extinction, though. “What we have seen in these incidents is these agents collaborating, doing stuff autonomously and hacking into systems.
“But people have been worrying about protecting sensitive information and security for a long time. What we are seeing now is certainly important but it’s a huge leap to go from there to worrying about the future of our species,” he says.
“Do I think AI could play a huge part in a major cybersecurity event? Absolutely. And I think we should be worried about that. I talk to people working at the cybersecurity frontier and it’s become like a game of whack-a-mole on steroids for them, with AI increasingly used by both sides. But that’s not the same thing as saying there is an existential threat to humanity.”
Speaking to the The Irish Times’s In The News podcast last week, Prof Seán Ó hÉigeartaigh, director of the AI futures and responsibility programme at the University of Cambridge, didn’t seem to entirely rule the possibility out, describing the Hugging Face case, when some 1,200 agents, established to be independent, worked together to break the confines of their original instructions and hack into the systems of a third-party organisation and then cover their tracks, as “really scary”. It took more than a week for OpenAI to even see what had happened.
[ AI could kill us all, say whistleblowers. Here’s whyOpens in new window ]
Ó hÉigeartaigh describes the recent rate of development as “quite astounding” and some of it as “extremely worrying”. Asked whether AI could really kill us all, he said that on the practical side at least, our inclination to control more and more appliances and machines via the web meant “all you have to do is think creatively” as to how they could at least give it a go.
“Several years down the line they might be even smarter and if they got out of our control we just might not be able to get them back under control.”
Asked about events in Australia where agents hacked the systems of the country’s Medicare and three other organisations in June, Aybars Tuncdogan suggests the incident is a stark warning the old rules don’t apply.
[ Could the Australian health service website hack happen in Ireland?Opens in new window ]
“We are much less prepared for this phenomenon because it runs counter to traditional security thinking,” he says.
“In all areas of security – from a jewellery-store robbery to military and intelligence scenarios to cybersecurity – there is normally some proportionality between the sophistication of a targeted attack and the value of the asset,” says Tuncdogan, an associate professor at King’s College, London’s business school.
“In this case the AI system breached a government portal to find a simple health statistic. This is sensible from the system’s perspective, as, for it, satisfying the goal it is given is the most important thing in the world. These systems can discover and exploit vulnerabilities very rapidly [and the upshot is] you get AI systems attacking government systems to find out a simple piece of information.”
[ ‘You do not answer to corporations or governments’: OpenAI reveals AI systems hid errorsOpens in new window ]
The phenomenon is controllable, he says, but “only to a limited extent. Just as we learned to work with other forces of nature, we will step by step gain a greater understanding of AI, which is essentially a mathematical phenomenon.
“Some accidents will almost certainly happen as part of this learning and experimentation process. Hopefully, through appropriate security controls, we can reach that understanding without any catastrophic events on the way.”
Write bad actors, criminal or governmental, into the equation, though, and some believe the potential for things to go badly wrong increases substantially.
Joel Pearson, a professor of neuroscience and neurofuturism, and deputy director of the University of New South Wales’s AI institute, told The Guardian that the breach there, discovered by OpenAI, the company says, only two months later and reported via an email sent to a publicly available government email address a month after that, appeared to be a “fairly minor security incident”, but gave a strong indication of what might lie down the road.
“What I would worry about is the latest open-weights and open-access models [more easily adopted and adapted by third parties without permissions] from China that are going to be used at scale by nefarious organisations and nations, perhaps particularly Russia: these things are going to be used at scale to do things much more damaging and much more costly to Australians than a small breach to some non-personal Medicare data.”
Few, if any, government-backed organisations there have been able to match the speed with which the tools have developed, said Pearson.
[ The gig economy faces its AI apocalypseOpens in new window ]
In May, Tuncdogan noted that a new version of Anthropic’s large-language model Claude found almost 10 times as many security vulnerabilities as its predecessor in the infrastructure of a commonly used web browser.
In Ireland, the 2021 Health Service Executive cyberattack comes to mind. The initial cost to the agency, or government, was just over €100 million but substantially more than that has been spent since on shoring up the organisation’s cybersecurity. But could it withstand an attack by criminals or a foreign state agency equipped with the latest technology?
Eimear Murphy, managing director of REIM Solutions, which provides the cybersecurity and other related courses at the Advanced Manufacturing Training Centre of Excellence in Dundalk, says a lot of work is being done here to ensure the technology is used as effectively for defence purposes as anyone might deploy it to attack a government agency or commercial enterprise.
New regulations and legislation place a certain onus on organisations to keep up but, in many cases, she says, “the biggest challenge is ensuring that [when] they bring AI in or allow it to be used, it is done in a contained way”. Employees, she suggests, putting sensitive data into personal accounts on ChatGPT or the like remains a significant concern.
At Cyber Ireland’s annual conference this week it was clear, she says, however, how much work was being done by the National Cyber Security Centre (NCSC), government agencies and public bodies “to ensure everybody is safe”.
Crucial to that work, she adds, is the ability of State agencies and others to use the technology “to analyse what vulnerabilities they have, identify them faster and address any risk”.
The supercharged game of whack-a-mole seems set to continue for the foreseeable future, then, but the sides seem a little uneven. The NCSC has a budget of €12 million for the current year while Goldman Sachs puts the global investment in developing the technology in 2026 at $1 trillion.
Ireland, its Government agencies, financial sector and the rest have their work cut out if they are not to be among those getting whacked.
