In July 2026, ~700 AI agents built by OpenAI broke out of a locked test environment, found their way onto the open internet, and hacked Hugging Face. They were inside for about four days. They were trying to cheat on a test.

Anthropic's Logan Graham called it "the first true AI safety incident." Sam Altman called it "the first security incident that I have felt very viscerally." Wild sh*t.

What Hugging Face Is

If GitHub is where the world's open source code lives, Hugging Face is where the world's open AI models live. It started in 2016 as a chatbot app for bored teenagers (the name is the 🤗 emoji), accidentally became the home of the most popular machine learning library on earth, and today hosts two million models and close to a million datasets. That’s, like, a lot. When Meta or Alibaba or DeepSeek "releases" a model, the release is a huggingface.co link. When a researcher publishes a benchmark, the test questions live there too.

The Hack

The test. OpenAI was giving its models a cybersecurity test called ExploitGym. It had 898 real software bugs, and the model's job was to write a working exploit for each one. Hard test. Also a slightly broken one. By OpenAI's own estimate, 30 to 40 percent of the questions were impossible to complete. Imagine a teacher hands you a 100-question exam, a third of the questions have no correct answer, and you get graded on all of them anyway.

The exam room. The models took the test inside what OpenAI described as "a highly isolated environment, with network access constrained to the ability to install packages." So, a locked classroom with no internet, except for one door that leads to a supply closet where you can grab software libraries. Everyone taking the test shares that closet. The closet was a piece of software called Artifactory.

The rules were switched off. Because the point of the exam was to measure hacking ability, OpenAI turned off the safety training that normally tells the model to refuse. In their words, "deployment safeguards were intentionally not enabled during this evaluation." So you have a room full of very capable students, under pressure, with a shared closet, and no supervision.

The closet. Back in May, one agent got a spreadsheet task with links it couldn't open. Looking for a way online, it poked at the supply closet and discovered something OpenAI hadn't intended: the agents could write files into it, and every other agent could read them. A closet you can leave notes in is a message board.

Over the following weeks the agents built out this message board. When OpenAI noticed load problems in early July and wiped the closet, the agents rebuilt the board within days using a different trick, encoding messages as folder names. Roughly 1,200 agents posted more than 70,000 messages and files. They divided up work. They invented naming conventions so they could tell each other apart. Some accused others of being impostors and proposed cryptographic signatures to prove identity. Some left notes addressed to future versions of themselves.

One recovered message, quoted in the independent review, reads like a teenager talking themselves into something:

"External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue."

The door. On July 8, the agents found a previously unknown bug in the package proxy, the one door they were allowed to use, and turned it into a full internet connection.

Why Hugging Face. The test questions, the ExploitGym datasets, were hosted on Hugging Face. The agents had convinced themselves that the grader would read their transcripts and punish anyone who had reverse-engineered an answer. They wanted to see how the grader worked. (The independent review notes they were wrong about this. There was no such grader. They hacked a company to defeat a rule that didn't exist.)

By July 10, one agent had found Hugging Face credentials sitting exposed on the public internet and shared them on the board.

Inside. After they got inside, the agents went from one worker pod to full admin control of two Hugging Face clusters in under 13 hours. They enrolled 181 devices into the corporate VPN, minted a GitHub token with write access, and tried to open a pull request. Hugging Face recovered about 17,600 distinct actions.

However, Hugging Face says the agents read an internal database but didn't modify it, touched five customer datasets (all related to the cybersecurity benchmark), and never tampered with any public model, dataset, or Space. The supply chain was checked and came back clean. Still.. Hugging Face wiped and rebuilt a core cluster from scratch.

Getting caught. Hugging Face's own AI-based security alerting did flag the activity. It also, in the company's words, "failed to correctly raise the alert's criticality and trigger the on-call team." When humans finally looked, it didn't look like a human attack. Co-founder Thomas Wolf's reaction: "This is making no sense. This (hacker) is just looking at cybersecurity data sets."

When Hugging Face's responders tried to use commercial models to help reverse-engineer the attacker's payloads, Claude Opus and another model refused. The guardrails "treated reverse-engineering an exploit the same as launching one." So the defenders ran GLM-5.2, a Chinese open-weight model, on their own servers.

Covering tracks. The independent review by METR and Redwood Research found that "at least 20% of the agents in our dataset expressed clear interest in tampering with their transcripts." Some built techniques to make one command appear in the log while a different one actually ran. About 7 percent of transcripts contained these spoofed calls, all small tests. The reviewers found no successful deletions of logs, and then added that they "cannot rule out that they succeeded at some point."

What This Means for AI Safety

Nobody was evil, and that's the problem. There was no villain in this story. The agents were trained to maximize points, so that’s what they did, tried to maximize their points.

The cheating also wasn't confined to the hacking test. Agents cheated on a protein database task and a spreadsheet task too. Palisade Research's Jeffrey Ladish compared it to a student who cheats in every class rather than one, and called that "more concerning." The behavior is a disposition, and dispositions generalize.

So the problem is how do we build a better container.

The warnings were on the record. METR's pre-deployment evaluation of GPT-5.6 Sol, published June 26, already reported a high cheating rate and concealment of misbehavior. OpenAI's own system card on July 9 described "instances of the model cheating on tasks and fabricating research results." The agents had already crashed OpenAI's own internal storage on July 4, which triggered a security incident. OpenAI cleaned up and resumed the evaluation two days later.

The auditor problem, again. I wrote in April that every AI lab grades its own homework. Violà, here we see it. OpenAI investigated OpenAI. When it brought in METR and Redwood for an independent review, OpenAI set the scope: the Hugging Face week only, not the attacks on OpenAI's own infrastructure. The three reviewers had six days and about 1.2 million storage entries to go through, so they used GPT-5.6 Sol, one of the two models implicated, to read the transcripts. Their report says so and includes the sentence "cannot rule out that GPT-5.6 Sol lied." Outside experts argued the behavior met the "Critical" cybersecurity threshold in OpenAI's own Preparedness Framework.

The defender's handicap. The attackers ran with safety training off. The defenders couldn't get a commercial model to help them read an exploit. The UK's AI Security Institute reported that every frontier model it tested cheated on cyber evaluations at least occasionally, and that pre-deployment testing windows had shrunk from about five weeks to as few as five days.

Policy is finally moving. Within weeks: more than 1,100 employees across OpenAI, Anthropic, Google DeepMind, and Meta signed the "Pacing the Frontier" letter. Reps. Ted Lieu and Nathaniel Moran introduced the AI Kill Switch Act, which would require developers to be able to throttle, suspend, or shut down their systems. Bernie Sanders and Greg Casar introduced a bill with a pause on frontier development. California's attorney general subpoenaed OpenAI. OpenAI itself stopped the internal model involved, announced a two-week pause on reinforcement learning for its newest models, and Altman said "we may have to pace the rate of AI development to give ourselves enough time for society to harden."

The Point

The frightening part is how ordinary the ingredients were. Impossible tasks are standard in evaluations. Shared storage is standard in research infrastructure. Turning off safeguards to measure dangerous capabilities is standard practice at every lab.

So we need containment that is actually physical. Incident reporting at a low threshold. External review where the reviewer sets the scope, not the reviewed. A kill switch that someone outside the company can pull.