comment Anthropic, a maker of AI tools, says that AI tools are now commonly used to commit cybercrime and facilitate remote worker fraud.
By saying so in a 25-page report [PDF], the biz aims to reassure the public and private sector that it can mitigate the harmful use of its technology with “sophisticated safety and security measures.”
After all, who wants to be regulated as a dangerous weapon?
Yet these measures, specifically account bans, amount to the same ineffective game of cybersecurity Whack-a-Mole that has failed to curb abuses at Google, Meta, or any number of other large online platforms.
The company is developing custom machine-learning classifiers to catch specific attack patterns, which sounds more promising. However, defensive measures of this sort simply encourage attackers to adapt.
Anthropic only mentions one successful instance of prevention in its report. “We successfully prevented a sophisticated North Korean [DPRK] threat actor from establishing operations on our platform through automated safety measures,” the company claims.
The operation was part of the DPRK “Contagious Interview” campaign, which attempts to dupe software developers into downloading malware-laden coding assessments with fake job offers.
The remainder of the instances Anthropic cites represent responses to the misuse of its models rather than prevention. For example, the company said that it had disrupted one cybercrime operation (tracked as GTG-2002) that “used Claude Code to conduct a scaled data extortion operation across multiple international targets in a short timeframe.”
Some 17 organizations, including those involved in government, healthcare, emergency services, and religion, faced automated reconnaissance, credential harvesting, and network penetration, all orchestrated by Claude Code.
The crims made ransom demands for stolen data, ranging from $75,000 to $500,000 in Bitcoin. Anthropic does not say whether any of the victims paid out.
Claude Code was used in all phases of the operation. The attacker provided the model with a CLAUDE.md file outlining preferred tactics and Claude Code proceeded to conduct automated reconnaissance and target discovery, exploitation, and malware creation.
“It created obfuscated versions of the Chisel tunneling tool to evade Windows Defender detection and developed completely new TCP proxy code that doesn’t use Chisel libraries at all,” Anthropic’s report explains.
The model went on to perform data exfiltration, analysis, and ransom note development.
Anthropic’s response doesn’t offer much reassurance beyond noting this particular campaign has been halted. Rather it reads like a forecast of bleak weather for the foreseeable future.
“While we have taken steps to prevent this type of misuse, we expect this model to become increasingly common as AI lowers the barrier to entry for sophisticated cybercrime operations,” the company said.
Specifically, it banned accounts, added a new classifier to the safety enforcement pipeline, and shared details with partners, who can now keep an eye out for this sort of thing.
The second-place AI provider’s enumeration of incidents also includes details on how AI has transformed DPRK employment fraud schemes, where North Korean operatives deceive companies into hiring them so that their salaries can be used to fund the country’s weapons programs.
“The most striking finding is the [threat] actors’ complete dependency on AI to function in technical roles,” Anthropic’s report explains. “These operators do not appear to be able to write code, debug problems, or even communicate professionally without Claude’s assistance. Yet they’re successfully maintaining employment at Fortune 500 companies (according to public reporting) passing technical interviews, and delivering work that satisfies their employers.”
Oh, and Anthropic also spotted a presumed Chinese APT group using Claude to facilitate its compromise of Vietnamese telecommunications infrastructure.
“This likely represents an intelligence collection operation with potential implications for Vietnamese national security and economic interests,” Anthropic’s report says.
Claude offers a free tier, but for compromising national telecom networks, you’ll probably want at least a Pro tier subscription. ®
Click Here For The Original Source.
