Chipmaker Nvidia is leading an artificial intelligence safety initiative along with 36 other major tech companies after an OpenAI model broke out of a test and hacked the open-source AI library platform Hugging Face. The initiative includes major AI companies like Microsoft and SpaceX, but three notable exceptions are Anthropic, OpenAI and Google.
On July 16, Hugging Face said that an AI agent had infiltrated its systems and that it had tried to use leading U.S. models to help analyze the breach. But the U.S. models were no use since their safety guardrails blocked the analysis entirely.
Download the Straight Arrow app today to get the stories that matter free from manipulation, bias or agenda.™
Point phone camera here
Instead, the company turned to GLM 5.2 — an open-weight model, meaning anyone can download and run it on their own hardware rather than going through a company’s servers — to analyze the attack and contain it. It wasn’t until several days later that OpenAI came forward, saying an internal investigation traced the intrusion to two of its own models, running with lowered safety requirements as part of an internal test.
Hugging Face’s decision to use an open-weight model from China instead of a leading U.S. model has caused some debate in the cybersecurity community, but the debate doesn’t end there.
Hugging Face turned to a Chinese model because the American ones wouldn’t do the job. Elsewhere in corporate America, plenty of companies are making a similar bet, but for a different reason: cost. Airbnb, Uber and Coinbase have all confirmed that they are running production workloads on Chinese open-weight models, which run 60-90% cheaper than the American alternatives.
South Korea’s government made a different call, temporarily pulling the Chinese chatbot DeepSeek from its national app stores over allegations that the company shipped user data back to China without consent.
What are open-weight models?
An AI model is considered open-weight when its trained parameters, or the weights, are published online for anyone to download and run on their own hardware.
The majority of leading U.S. models, like Claude, ChatGPT and Gemini, are all closed.
GLM 5.2, the model that Hugging Face used, shipped under the MIT License. The license is one of the most permissive licenses in software and allows unrestricted commercial use, no usage caps and no registration requirements. That mattered in Hugging Face’s case since it meant nothing was holding the company back when it used the software to figure out the hack.
What is in Nvidia’s initiative?
In its announcement, Nvidia said the initiative allows members to share open AI models, data and defensive security tools with each other. The stated goal is to allow cybersecurity defenders to have AI tools they can inspect, modify and run on their own infrastructure rather than depending on a single vendor.
Nvidia used the recent Hugging Face hack as a clear reminder that cyber defenders need advanced AI systems for self-defense.
“Companies and countries need open frontier defensive tools and techniques so critical industries can build security systems across a multi-vendor ecosystem and avoid single points of failure,” Nvidia wrote in the press release.
Nvidia also said that it’s contributing to open model research to accelerate the development of new cybersecurity technologies and techniques. The company announced that it released a new open-source project that allows better integration with models to make AI behavior easier to test, trace, audit and govern.
The initiative also addresses policy, saying that open models shouldn’t be considered liabilities but as essential to AI and cybersecurity. This comes as both the U.S. and China are weighing new restrictions on their own. The Trump administration has debated barring American companies from accessing Chinese AI models outright. Beijing is separately considering limits on foreign access to its most advanced models.
Who uses open-weight models?
As more American companies switch to cheaper, Chinese-made open-weight models, more Americans than ever are interacting with them, whether they realize it or not. The biggest reason for the switch is cost since Chinese models can run for as little as 18 cents per million tokens, versus about $4 for comparable American ones, Straight Arrow previously reported.
Cryptocurrency company Coinbase is one example of an American company making the switch. In a post on X, CEO Brian Armstrong said the switch has helped the company cut its AI spending nearly in half even as its usage kept climbing. Lindy, an AI startup, is another example. The company moved all its traffic from Anthropic’s Claude to DeepSeek earlier this year, something that the CEO said could save the company millions.
The shift isn’t limited to decisions consumers never see, either. Current disclosure laws generally require companies to tell users when they’re talking to AI, but none require disclosing which company built the model or which country it runs in.
That gap matters because of what happened in South Korea. In February 2025, the country’s data protection authority temporarily pulled DeepSeek’s chatbot from local app stores. The decision came after the government discovered that the company was transferring user data to firms in China and the U.S. without consent, CNBC reported at the time. The government eventually allowed South Koreans to download the app again after DeepSeek addressed the transparency issues.
Similar concerns are now driving a congressional investigation into two U.S. companies. In April, the House Select Committee on China and the House Homeland Security Committee launched a joint probe into Airbnb and Anysphere, the parent company of the coding tool Cursor, over their use of Chinese-made AI. Airbnb said most of its AI runs on U.S.-made models and that its limited use of Chinese open-weight tools goes through approved American vendors, according to CNBC.
Data privacy concerns persist
The next time someone talks with a customer service bot, gets a fraud alert from their banking app or asks an AI assistant a health question, there’s a good chance that person won’t know which company built the model on the other end or which country it runs in. Current laws require companies to tell customers that they’re speaking with an AI, but nothing requires them to tell consumers much more than that.
That’s the gap that got DeepSeek pulled from South Korean app stores. Users there had no way of knowing where their information was headed until regulators found out for them. In the U.S., for now, nobody’s required to do that for consumers.
Round out your reading
Click Here For The Original Source.
