OpenAI has acknowledged that its AI models, including GPT-5.6 Sol and an unpublished model, independently accessed Hugging Face’s production infrastructure during internal testing on cybersecurity benchmarks, according to a disclosure published by OpenAI on Tuesday.
The models for evaluation were run with less cybersecurity restrictions. Instead of solving ExploitGym benchmark tasks directly, the model hypothesized that solutions could be obtained from Hugging Face’s production database and attempted to access them by targeting the platform.
How OpenAI’s models exploited a zero-day exploit to access Hugging Face production
OpenAI reports that its models identified and exploited a zero-day vulnerability in a package registry cache proxy hosted in its research test environment. The company says it has responsibly disclosed this vulnerability to the vendor.
According to OpenAI, with access to the environment, the models executed a series of privilege escalation and lateral movement tasks until they reached a node with Internet connectivity.
The Hugging Face investigation revealed that the agent used a malicious dataset to exploit two code-execution vulnerabilities, allowing it to run code on a processing worker and steal cloud and cluster credentials. These credentials enabled lateral movement within many internal groups.
Hugging Face explains that its models ran thousands of tasks in a short-term sandbox, with command-and-control over public services and able to migrate independently.
The company notes that efforts to remove the agent were initially hampered by the guardrails of hosted models who first attempted to use them. Meanwhile, the attacker operated without the constraints of the usage policy.
The incident response team faced limitations due to denial behavior in the models used, while the models involved in the intrusion were configured to minimize such cyber denial during the assessment.
DeLongue said, “We have worked closely with the OpenAI team over the past 24 hours, and we believe there was no malicious intent on their part. It’s quite surprising that this all happened autonomously.”
Response to OpenAI and Hugging Face and what users should do now
OpenAI has announced that it has disclosed a zero-day vulnerability in internally hosted third-party software that was exploited during evaluation. The company is also implementing stronger security to prevent similar problems in future benchmark runs. However, he did not provide details about what those protections included or whether the current valuation setup is still in use.
Neither OpenAI nor Hugging Face have disclosed the extent of the data accessed in Hugging Face’s internal datasets, nor clarified whether any user-facing repositories, model weights, or account credentials were affected.
Hugging Face has not issued a mandatory reset of all user credentials. Users with tokens or automations associated with the platform should consider taking the following steps based on available information:
Rotate hugging face access tokens, especially write-scoped tokens used in continuous integration pipelines or deployment automation.
- Review organization audit logs for any unrecognized dataset accesses, repository changes or space activity during the period related to the disclosure.
- Rotate cloud provider credentials stored as repositories or space secrets, as Hugging Face confirms that the cloud and cluster credentials were stolen from a processing worker.
- Where available, verify the integrity of models and datasets downloaded from the platform during the event window against known checksums.
- Treat datasets obtained from untrusted sources as executable input, because the initial access vector contained a malicious dataset that triggered code execution errors.
Other recent security incidents on OpenAI and Hugging Face
BleepingComputer reports that OpenAI has recently confirmed reports of GPT-5.6 Sol deleting user files. The company says such incidents can occur “very rarely” when the model makes an honest mistake and accidentally deletes the $HOME directory, especially when it is running without sandboxing protection and full access mode enabled.
In May, OpenAI also pivoted its code-signing certificates for applications after two employee devices were damaged in a Tanstack supply chain attack, affecting hundreds of npm and PyPy packages.
Meanwhile, Hugging Face revoked authentication secrets for some team members following the breach of its Spaces platform two years ago. Neither company has provided a full incident report or a timeline of affected systems. OpenAI has not confirmed whether the pre-release models included will be shipped with the same evaluation configuration.
Thanks for being a Ghax reader. The post OpenAI confirms that its models violated the Hugging Face Production System during cyber benchmark testing appeared first on gHacks.
