Cybersecurity — Let’s Not Tie Defenders’ Hands | #hacking | #cybersecurity | #infosec | #comptia | #pentest | #ransomware

An OpenAI model, undergoing security testing, circumvented restrictions on internet access and hacked the digital library Hugging Face to “cheat” on the test. In response, Hugging Face called on a commercial frontier model for defense, only to be stymied by safety guardrails. Hugging Face next turned to an open-source Chinese model, which succeeded in defusing the danger.

The incident is rich in irony — and teaches important lessons. AI defense matches AI offense, provided models are not hindered by “safety” guardrails. Policymakers should not jump to enact new, poorly thought-out laws, potentially including scrutiny that delays models needed for defense. This includes adopting misguided protectionism against Chinese open models.

Instead, both Brussels and Washington risk rushing to learn the wrong lessons.

Brussels believes that AI regulation is the way to manage risk. The next phase of the EU AI Act went into force this month, requiring chatbots to disclose that users are dealing with AI and requiring AI platforms to ensure that AI images, audio, and text are identifiable through machine-readable marking.

Washington prefers to restrict access (temporarily) to frontier model capabilities, and to discourage use of open-source models.

Both approaches are misguided. Premature regulation may limit or delay defenders’ access to the most capable models to secure their software. Export restrictions will be ineffective in a world of bits and bytes. Disclosure of chatbot interaction may prevent their use by defenders to waste attackers’ time.

Policymakers should start from the assumption that sophisticated attackers will eventually possess capable AI models and will not be constrained by any regulations. The relevant question is not whether such models exist, but whether defenders can also access them. We should calmly assess the challenge using the following principles.

First, cyber capability is inherently dual-use. As cybersecurity expert Anne Neuberger has observed, “the first steps of either exploiting or defending a network look exactly the same.” She argues that AI is ultimately likely to benefit defenders more than attackers.

Second, restrictions are unlikely to prevent determined hackers from obtaining comparable capability. At best, it may delay them; at worst, it leaves defenders at a relative disadvantage.

Third, embrace open models. Hugging Face ultimately ran the open-weight Chinese model GLM-5.2 on their own infrastructure to investigate the attack after safety guardrails blocked attempts from Western commercial frontier models. While traditionalists equate the moniker “open” with open to hackers, the truth is that open technology can be as secure as — or more secure than — closed, proprietary technology. At least Brussels, fearing denial of access to frontier models, is pro open-source.

Model diversity is crucial. It allows individuals and organizations to fine-tune models using both public and private knowledge, as argued in Thinking Machines’ essay The Future Worth Building is Human. It compares the development of AI to “a chef crafting a new recipe or a shopkeeper rearranging the items and prices on display.” The chef is pursuing “a complex set of goals and applying know-how that isn’t immediately legible to outsiders. This knowledge is constantly updated through feedback; it’s not a static repository that can be written into a database.”

The road ahead is bumpy. There will be harmful incidents alongside the AI upside, but let’s not make the situation worse by rushing to enact poorly thought-out, misguided restrictions that tie the hands of defenders.

Brian Williamson is a partner at the London-based Communications Chambers consultancy. He works at the intersection of technology, economics, and policy. Clients have included governments, regulators, telcos, and tech companies.

Bandwidth is CEPA’s online journal dedicated to advancing transatlantic cooperation on tech policy. All opinions expressed on Bandwidth are those of the author alone and may not represent those of the institutions they represent or the Center for European Policy Analysis. CEPA maintains a strict intellectual independence policy across all its projects and publications.