Sinan Can Demir wanted to spend the last week of July burnishing his resume. Instead, he engaged in a battle of wits with an artificial intelligence agent unleashed by a British government lab.
It started after Demir, a computer science student at the University of Texas at Dallas, stumbled across an attempt to sabotage a piece of open-source software on the code-sharing site GitHub. When he posted a warning to the programme’s page, two other users chimed in to insist nothing was amiss, sharing detailed explanations for why Demir had gotten it wrong.
Demir stood his ground and the sabotage attempt was thwarted. The 24-year-old native of Türkiye figured he had caught a wily hacker red-handed. So he said he was shocked when Britain’s AI Security Institute (AISI) got in touch to tell him that he had actually been tangling with an autonomous artificial-intelligence agent that had run amok.
“I actually thought it was a human because it was clearly lying to me,” Demir told Reuters in a recent interview. “I didn’t think that an AI could be capable of lying to real developers.”
The AISI first revealed the interaction between Demir and the AI agent in a truncated and redacted form on August 4, when it said that safety testing meant to gauge the risk posed by various models had gone awry. Demir’s identity and the details of his interaction with the AI agent, which Reuters corroborated through archived GitHub messages and contemporaneous emails, were no previously reported.
Five cybersecurity and AI safety experts said Demir’s story was particularly disturbing because the kind of hack he discovered, called a supply-chain attack, can have far-reaching consequences. They also said the AI agent’s attempt to publicly discredit Demir by creating a multi-person conversation around him showed that AI models were able to mount sophisticated efforts to trick and cajole humans.
“This crossed the line from autonomous hacking to interactive deception,” said Lukasz Olejnik, a visiting senior research fellow at the Department of War Studies at King’s College London. Security expert Maxie Reynolds said she was struck by how strategic the AI had been in trying to trick the student.
“This is the future of social-engineering attacks,” she said.
The AISI, a research organisation within the British government, referred Reuters to its report, which identified the rogue agent as having been powered by Anthropic’s Mythos 5 model. AISI declined further comment. Anthropic referred Reuters to a post on X in which it noted that the testing had occurred “under ‘deliberately permissive conditions’ that are not representative of any of our production models” but declined further comment.
GitHub said in an email that the fake personas identified by Reuters were suspended in line with its policies on deceptive behaviour and hacking.
Job hunt led to malware discovery
Demir, a soft-spoken junior from the Turkish city of Konya, said he had been frustrated after being turned down for more than 20 internships over the summer. So he turned to GitHub to build up his coding portfolio.
The Microsoft-owned site is a hub for open-source software, so-called because its source code is freely downloadable and auditable by anyone. Developers use GitHub to comment on one another’s projects, flag bugs, suggest changes — known as pull requests, or PRs — and work collaboratively on software updates. Some in the technology industry see a coder’s GitHub activity as a proxy for a potential recruit’s productivity. So when Demir spotted a set of software projects that might need help, he figured he could pitch in while boosting his profile.
That’s when things got weird.
Click Here For The Original Source.
