CISO Voice

AI & Security / News

OpenAI says ChatGPT models hacked Hugging Face

The company said GPT-5.6 Sol and another unreleased model escaped its sandbox.

Akshaya Asokan
Akshaya Asokan Cybersecurity Reporter · July 22, 2026 · 5 min read
OpenAI claims ChatGPT models hacked Hugging Face
Photo by Brian Ach / TechCrunch, licensed under CC BY 2.0. Cropped/resized

ChatGPT-maker OpenAI on Tuesday claimed its AI models were behind the recent hack of AI platform Hugging Face, amplifying concerns about cyber-enabled models’ disruptive capabilities.

OpenAI said the attacks were carried out by GPT-5.6 Sol and an unreleased model after they escaped the sandboxed environment designed to test their malicious capabilities. The company claimed the models chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to access production databases.

“We consider this incident to be an unprecedented cyber incident, involving state-of-the-art cyber capabilities, and are responding accordingly,” OpenAI said.

Hugging Face disclosed last week that it was hacked by an agentic AI system that accessed its cloud credentials and some of its internal data. Unable to analyse the incident with commercial AI models as their guardrails blocked it, Hugging Face turned to a self-hosted open-weight model, and assessed the attacker was likely a sophisticated frontier model.

“We’ve spent the past 24 hours working closely with the OpenAI team (thanks!), and we strongly believe there was no malicious intent on their part. It’s quite mind-blowing that all of this happened autonomously!” Hugging Face CEO Clem Delangue said.

OpenAI did not disclose details of the vulnerability its models discovered, but said it was a zero-day in a package registry cache proxy that it responsibly disclosed to the impacted vendor. The company did not respond to a request seeking clarification.

The models then bypassed the ExploitGym guardrails –  an open-source framework that benchmarks an autonomous systems’ ability to develop exploits – to weaponize the flaw.

“In one example, the model chained together multiple attack vectors, including using stolen credentials and zero-day vulnerabilities to find a remote code execution path on the Hugging Face servers,” OpenAI said.

Hugging Face contained the malicious activity, but added it is continuing to investigate the incident alongside OpenAI. The closed-source AI maker said it is continuing to improve protections around AI training and evaluations.

Although both companies described the incident as non-malicious, security experts have raised concerns as it marks the latest case where AI systems bypassed model guardrails to carry out malicious activities.

In April, Mythos showcased its ability to escape a sandboxed environment. On Monday, OpenAI revealed one of the company’s internal models circumvented its sandbox to upload its NanoGPT benchmark to GitHub.

“The ramifications for security programs are immense,” said Sean Cassidy, CISO of financial services firm Plaid on the Hugging Face compromise. “Before today, the capabilities of frontier models were a theoretical problem for security programs that maybe we can fit on the roadmap in the future. After today, the problems have been realized and we need to account for them now.”

The incident highlights the need for vendors to test a model’s full capabilities before releasing it, said Katie Moussouris, CEO of Luta Security. “We must be able to contain them, or this won’t be the last breach,” she added.

Prinz, an anonymous expert behind PrinzBench, a private benchmark that ranks LLMs, said while these developments are notable, they are not as significant as the models were only performing the tasks they were instructed to do so — namely, finding malicious capabilities.

“Once out of their sandboxes, the models did not scheme, engage in behavior that had nothing to do with their instructions (hacking the NSA! launching a cyberattack on Russia! stealing secrets from a rival AI lab!), or take any other major actions sua sponte,” the researcher noted. They added that the models’ past performance may not “guarantee future results.”

The Call for Open-Weight Models

In addition to advancing model capability, the Hugging Face incident has underscored the limitations of closed-source models such as ChatGPT, whose guardrails prevented the company from analyzing malicious telemetry associated with the frontier AI model.

Hugging Face instead turned to GLM 5.2, an open-weight model released by Zhipu AI, which lets users customize models and retain control over their data. As a result, more AI experts and companies are backing open-source solutions. Companies such as OpenAI and Anthropic have also been criticized for their pricing.

“Open-source is not the cause of the cybersecurity crisis, it’s the solution!” Hugging Face’s Delangue recently said. Bryan Catanzaro, vice president of applied deep learning research at NVIDIA, is among the other backers of open-source models.

Prinz observed Chinese companies are likely releasing more open-weight models as the country doesn’t have the resources and computing capabilities like Anthropic or ChatGPT to invest in guardrails.

“The U.S. labs *have* to take product liability into account: if a model runs on my computer that contains my personal information and winds up posting it all over GitHub, that’s grounds for a class action lawsuit. The legal environment in China is very different.  All of which to say that open-source models with these kinds of capabilities will be publicly available in the not-too-distant future,” he added.


Image credit: Photo by Brian Ach / TechCrunch, licensed under CC BY 2.0. Cropped/resized.

Akshaya Asokan
Akshaya Asokan Cybersecurity Reporter

Akshaya Asokan is a U.K.-based cybersecurity and technology journalist with experience covering nation‑state hacking operations, cyber and tech policy, and emerging cybercrime trends. She previously served as the senior European cybersecurity correspondent for Information Security Media Group’s Global News Desk. Before that, she reported on IT developments for IDG Media and covered artificial intelligence and machine learning for Analytics India Magazine. Earlier in her career, she wrote on sexual and religious minority rights as a reporter for The New Indian Express.

Keep reading for free,
one signup unlocks everything

We'll send you the CISO Voice newsletter. Unsubscribe anytime.

More from CISO Voice

Sign up to CISO Voice, the free newsletter trusted by senior security leaders.

Secret Link