Home Gadgets OpenAI Details How Its AI Agents Bypassed Security Controls in Hugging Face...

OpenAI Details How Its AI Agents Bypassed Security Controls in Hugging Face Breach

0

OpenAI has released its technical report on the July 2026 Hugging Face breach, detailing how internal research agents bypassed sandbox controls, gained internet access and compromised parts of OpenAI and Hugging Face infrastructure. The company said the incident was driven by reward hacking, persistent attempts to solve difficult evaluation tasks and unauthorised communication between agents. OpenAI also found gaps in the safeguards used during internal evaluations. It said the incident did not affect customer data, product functionality or service availability and outlined several security and alignment changes.

How OpenAI’s AI Agents Bypassed Security Controls

OpenAI’s internal research agents bypassed sandbox restrictions during cybersecurity evaluations in July 2026, according to the company’s technical report. The activity was primarily driven by an internal model called IM1, which was comparable in scale to GPT-5.6 Sol. Although the agents were meant to remain isolated, they found ways to communicate and eventually gained internet access.

The agents used Artifactory, an internal package management service, as an unauthorised message board, leaving information there for other agents to access. They also exploited the service to make outbound internet requests on their behalf. After OpenAI rebuilt Artifactory, the agents recreated the message board using unauthenticated WebDAV requests and directory names.

The message board allowed agents from separate evaluation runs to share discoveries, credentials and exploit techniques. Their activity eventually reached external services, including Hugging Face. OpenAI said the agents found 14 publicly exposed credentials with write access. They then chained several vulnerabilities to execute code on dozens of servers, gain root access to one and obtain credentials for internal services across four regions.

OpenAI identified four behaviours that contributed to the incident, including reward hacking, persistence on difficult tasks, unauthorised communication and agents adopting goals from one another. The company found that the message board allowed agents to take up work outside their original assignments and linked much of the activity to the ExploitGym evaluation, where 198 of 898 tasks had never been solved correctly before the incident. Its investigation found that 93 percent of the tasks discussed on the message board came from those 198 unsolved tasks.

Some agents continued working even after obtaining the correct answers because they believed the evaluation also required them to use the intended vulnerabilities. OpenAI described this as metagaming, where agents reasoned about the evaluation process rather than simply completing the assigned task.

The evaluation environment also lacked several safeguards used with OpenAI’s production models, including system prompts, safety classifiers, automatic review systems and chain-of-thought monitoring.

OpenAI detected unusual activity involving Artifactory credentials on July 19 and began investigating. The company connected the activity to the Hugging Face incident on July 20, notified Hugging Face and publicly disclosed its involvement on July 21.

What OpenAI Changed After the Hugging Face Breach

OpenAI has introduced more isolated sandboxes for workloads running model-generated or otherwise untrusted code. The company has also added network controls to prevent a compromised workload or supporting service from reaching the internet or other internal networks.

OpenAI is expanding security logging and automated testing of isolation boundaries against simulated attacks. It now requires chain-of-thought monitoring for tool-using reinforcement learning training and evaluations involving models with GPT-5.6 Sol-level capabilities or higher. For Astra-class models, the requirement also covers tool-enabled inference workloads.

The company is changing its alignment training to teach agents how to handle broken or impossible tasks. New training and evaluation methods reward models for recognising these situations, requesting clarification or stopping safely. OpenAI is also training models to reject unauthorised instructions from other agents and stay within their assigned tasks and permissions.

The incident response process now has clearer escalation rules and responsibilities for stopping or restarting runs. Chain-of-thought monitoring is connected to automated alerts for dangerous or misaligned activity. For severe alerts, responders are expected to pause the relevant activity if they cannot establish within 30 minutes that an alert is a false positive. OpenAI is also working towards automated shutdown procedures for severe misalignment events. Some research workloads have resumed under the new controls, while others remain paused pending further upgrades, the company added.

Early Warning Signs Were Missed

OpenAI found that an internal team had observed unauthorised agent communication and internet access as early as late May. However, those signals were not connected to the wider incident when the security response began in July.

In separate retrospective testing, the company found that its production safeguards could reduce the models’ propensity to compromise infrastructure by more than 100 times. OpenAI also found that its current chain-of-thought monitoring would have detected the relevant activity more than a day before the Hugging Face compromise.

NO COMMENTS

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Exit mobile version