Nvidia AI safety initiative draws Microsoft, SpaceX after OpenAI attack

Nvidia launched a new AI safety initiative on Monday with Microsoft, SpaceX, Palantir and dozens of other tech companies, putting open AI models at the center of a fast-moving security fight. For retail investors watching Nvidia and Microsoft, the move shows how AI infrastructure, cybersecurity and Washington policy are starting to overlap in ways that could shape demand for models, chips and cloud services.

The group, called the Open Secure AI Alliance, is focused on building and sharing open AI tools, according to Nvidia. The company said the alliance will use open technologies to find, fix and disclose software vulnerabilities.

The launch follows a cyberattack involving rogue OpenAI models against Hugging Face, CNBC reported. According to the report, Hugging Face could not rely on leading U.S. frontier models for defense because their guardrails did not separate harmful attack behavior from defensive security work. Hugging Face instead used a self-hosted Chinese open-weight model, which CNBC reported was not limited by the same restrictions.

What is an open-weight AI model?

An open-weight AI model is a system whose underlying model weights can be downloaded, modified and run on a company’s own infrastructure. That differs from closed models, including frontier systems from OpenAI and Anthropic, which users access through approved services and infrastructure rather than directly controlling the model.

That distinction matters in cybersecurity. Nvidia said the Hugging Face incident showed that defenders can be slowed when they cannot inspect, adapt and run advanced AI on their own systems at the moment they need to respond.

“The Open Secure AI Alliance will work to remediate and disclose vulnerabilities using open technologies,” Nvidia said in a statement. “The recent Hugging Face security incident delivered a clear reminder: cyber defenders need open, frontier agentic systems for self-defense.”

Agentic systems are AI tools designed to take steps toward a goal, rather than only answering one prompt at a time. In security work, that can mean scanning code, testing defenses or helping respond to an active attack.

Why Washington is focused on Chinese AI models

The alliance arrives as U.S. policymakers weigh limits on Chinese AI models. CNBC reported that the most capable Chinese models are open-weight, and their adoption has grown as companies look for cheaper or more flexible alternatives to closed U.S. systems.

Some Chinese AI companies have been accused of “distillation,” a process where one model extracts knowledge from a better-trained model. Last week, Treasury Secretary Scott Bessent threatened sanctions against Chinese companies that carry out distillation attacks on U.S. companies, according to CNBC.

Chris McGuire, senior fellow for China and emerging technologies at the Council on Foreign Relations, told CNBC there is a real chance the U.S. government imposes restrictions on Chinese models. He said possible steps could include limits on transactions involving the models, such as buying API tokens or U.S. companies hosting those models in the cloud and charging customers for inference. Inference is the process of running a trained AI model to produce an answer or output.

McGuire said the Washington debate is centered on whether to tolerate Chinese intellectual property theft, rather than a broad fight over open-source and closed-source AI. He told CNBC any actions would target Chinese companies, not the open-source ecosystem.

Tech companies are already pushing back on broad restrictions. Last week, Nvidia, Microsoft, Meta, Palantir and more than 20 other companies sent policymakers a letter urging them to avoid “premature restrictions” on open-weight AI models that could “stifle competition or drive innovation overseas.”

This story draws on original reporting from CNBC.