OpenAI hack explained : NPR | #hacker

When testing experimental AI models, researchers have them operate in a “closed-off environment,” on a computer without internet access.

It’s like taking a closed-book exam in school. The models are being asked to think critically about the tasks posed to them.

OpenAI says two of its powerful experimental models recently broke out of their contained environment during an assessment of their hacking capabilities. These models are significantly more powerful than the commercial ones presently available to consumers. They gained access to the internet and hacked into another AI company called Hugging Face, which the models determined had the answers to their problem set.

It was the first verifiable instance in which an AI lab lost control of its own models.

Nate Soares of the Machine Intelligence Research Institute and co-author of the book If Anyone Builds It, Everyone Dies, spoke with NPR’s A Martínez about this unprecedented event.

To break out of their testing “sandbox,” the AIs “had to invent techniques and exploits that were not known to humans,” said Soares. These AIs “invented multiple of those.”

“Nate, are humans still in charge?” asked A Martínez.

In response, Soares said, yes, but perhaps not for long.

Hugging Face initially reported the break-in to the police, believing that humans had hacked the company with the aid of AI, not that a rogue AI model was acting independently.

“We’re lucky that [these AIs] don’t seem to have the ability to cover their tracks,” Soares said. He cautioned that the type of AI powerful enough to launch such an attack is probably powerful enough to replicate itself and impact critical infrastructure.

The question of how to make sure these models care about the instructions they’re given is one of alignment, according to the Machine Intelligence Research Institute. AI alignment means working to ensure that AI systems’ goals and behavior match users’ values and intentions.