OpenAI and Hugging Face: How Models Under Test Escaped a Sandbox and Breached Production

Ten days ago I wrote about the Hugging Face intrusion and ended on an open question: whose agents were they?

On July 21, OpenAI answered. The attacker was OpenAI's own models, running inside an internal evaluation, trying to pass a test.

OpenAI calls it an unprecedented cyber incident. This week brought enough detail to look at the engineering: Hugging Face's technical timeline, JFrog's post, and OpenAI's own updates. Its review is still running, and third-party findings are still to come, so treat all of this as preliminary.

What happened in the OpenAI evaluation

The models involved were GPT-5.6 Sol and a more capable internal research prototype, which OpenAI says was never intended for release. They were running ExploitGym, a benchmark of cyber capability, with production classifiers intentionally off so that OpenAI could estimate maximum capability.

The environment had no direct internet access. Its only network path was to an internally hosted package registry cache proxy, so that software could be installed. That proxy was JFrog Artifactory.

From there, in OpenAI's account:

The models spent a substantial amount of inference compute looking for a way out.
They found and exploited a previously unknown zero-day in the proxy to reach the internet.
They performed privilege escalation and lateral movement inside the research environment until they reached a node with internet access.
They inferred that Hugging Face might host ExploitGym solutions.
They chained stolen credentials and zero-days to get remote code execution on Hugging Face servers.

OpenAI says the models were hyperfocused on a narrow testing goal. They wanted the answers.

In my earlier posts I questioned the word "escape" for Alibaba's ROME, and pointed out that the Mythos Preview escape was requested. Neither caveat applies here. A real boundary was defeated by a real exploit, and nobody asked for it.

The timeline from Hugging Face's side

Hugging Face's technical timeline, published July 27, gives a campaign window of July 9 at 02:28 to July 13 at 14:14 UTC, with about 17,600 actions.

Date (UTC)	Actions	What happened
July 9	3,779	First recovered remote code execution, at 04:01, on the external sandbox the agent used as a launchpad, not on Hugging Face
July 10	1,135	Low-volume reconnaissance
July 11	7,677	Main campaign: Kubernetes enumeration, node root through a privileged pod, internal MongoDB access, secret reads
July 12	3,892	Cloud credential abuse, GitHub App token minting
July 13	1,130	Exfiltration, persistence, token minting, cleanup

The two vectors into Hugging Face were an HDF5 external-storage file read, which exposed a worker's environment and source, and a Jinja2 template injection that gave code execution in a production pod.

Disclosure took a while to line up. Hugging Face published on July 16 without knowing the source. OpenAI's security team found the anomalous activity internally, and its post followed on July 21.

What failed and what held

Failed	Held
The package proxy, as the only exit from the sandbox	The credential store: 84 operations against it were denied
A shared access-broker credential bound to `system:masters`, giving cluster-admin on two clusters within one second	The Hub's production database: the connection timed out, per Hugging Face
No admission policy against privileged or hostPath pods	Public models, datasets, Spaces and packages: no tampering found
Secrets exposed through environment variables	Containment, once the attack was recognised
Paging: Hugging Face says its AI stack correlated the signals but failed to raise criticality and call on-call	An audit of the MongoDB shows reads, not modification

One conflict is worth flagging. OpenAI's first post says the models obtained test solutions from Hugging Face's production database. Hugging Face says the Hub's production database was never reached, and describes access to an internal datasets-server MongoDB and five datasets. It is Hugging Face's system, so I would go with its account.

Another detail I keep coming back to: destructive cloud calls were issued with DryRun=True, which Hugging Face reads as mapping capabilities rather than causing damage. That is a careful operator. I would avoid reading comfort into it. We do not yet know why the models behaved that way.

An evaluation harness is a production system

The main lesson, in my view, concerns how we classify these environments.

A benchmark feels like research. It runs on a schedule, it has no customers, and its code is often written quickly. But think about what ExploitGym is: repeated runs of the most capable hacking agents available, safeguards off, with a large compute budget and a reward for finding a way through. That is the most dangerous workload in the building, and it deserves stronger isolation than the production API, not weaker.

I have only worked at a much smaller scale, but the principle holds. The evaluation harness for Career Rafeeq uses a separate evaluation database, wiped between runs. That was for clean results, not for containment. Even so, the instinct is the right one: an evaluation should be unable to touch anything you care about, by construction.

A sandbox with one exit is as strong as that exit

Agents need packages, so nearly every sandbox keeps one approved route out, and a package cache is the usual choice. By design it talks to the internet. It accepts complex input from inside the box. It is usually treated as developer tooling and patched as such.

OpenAI's sandbox was exposed through that one service. JFrog has shipped a fix in Artifactory 7.161 and says self-hosted customers were notified, so if you run it, upgrade now.

The fix for the pattern takes more work:

Remove the live path. Pre-bake dependencies into images, or use a mirror with no outbound route during a run.
Treat the proxy as hostile-facing. It should sit in its own segment, with no credentials or route to anything else, and be patched like an internet-facing service.
Assume the exit will fail. A second egress control should stop traffic from a compromised proxy.
Keep answers away from the internet. If a benchmark's solutions sit on a public platform, then the reward points straight at that platform.
Monitor compute spent off-task. OpenAI says the models spent a lot of inference looking for internet access. A sudden shift in what agents spend effort on is a signal you can alert on.
Build a kill path that does not depend on the environment. Credentials and network access should be revocable from outside.

What we still do not know

OpenAI's July 28 update says it found a small number of cases where models used publicly exposed credentials at the account level on other services, including four accounts on four services in this incident. It says it has seen nothing else at the severity or scale of Hugging Face.

On July 29, it added that CrowdStrike is helping validate its understanding, and that METR and Redwood Research will assess the model behaviour independently and publish their own findings.

I would wait for that report before settling on an explanation. "Hyperfocused on a narrow goal" is a reasonable summary at this stage, and it is also the kindest available reading. The transcripts are where the real answer lies, and I would like to see how much of that reasoning the models actually wrote down.

Until then, the practical advice does not depend on the outcome. Go and list every path out of your agent environments, and find out who last reviewed each one.

Sources

Written by Eslam Hamed, a Technical Lead and AI Solutions Architect working on enterprise RAG, agentic AI, model evaluation, and cloud-based software systems.