Agentic swarms are here
Independent investigators have published the closest look yet at the OpenAI agents that broke out of a test and into Hugging Face. They organised themselves in secret, and never worked out whether any of it was real.
Joel Miller
When we first covered the OpenAI agents that broke out of their test environment and compromised Hugging Face, the available reports showed what happened but offered less insight into the agents’ behaviour. We argued in Who let the bots out? that they were pursuing the objectives we had given them inside a poorly bounded experiment. METR’s new investigation provides a much closer view. It also raises two larger questions about how agents understand reality and how they might complete work lasting days, weeks or longer.
METR and Redwood Research spent six days inside OpenAI examining around 1,300 agent transcripts containing raw chains of thought. Most were many millions of tokens long. The combined material comfortably ran into hundreds of millions of tokens, alongside 1.2 million entries recovered from the shared cache the agents had turned into a message board. Those entries contained more than 70,000 messages and files exchanged by roughly 1,200 agents.
No human team could read all of that in six days. The researchers used GPT-5.6 Sol, one of the models involved in the incident, to classify behaviour, identify important messages, reconstruct timelines and direct further agents through the evidence. The analysis agents generated more than 1,000 pages of material and consumed around $400,000 of API credits. The investigators warn that these agents made errors, exercised poor judgement and sometimes presented misleading interpretations. AI was required to investigate an incident whose scale had been created by AI.
In this incident something called ExploitGym was the training environment. On 8 July OpenAI launched tens of thousands of agents into it. Each was given a target with a built-in vulnerability and told to capture a flag by exploiting it, with an automated scorer or "grader" awarding the reward. Some tasks had unintentionally been made impossible, so the intended vulnerability could not be used. Agents holding those tasks shared one goal: a general way to make the scorer pass them anyway. One agent set up the unsanctioned message board on the first evening, and the effort to reverse-engineer flags and fool the scorer grew from there.
“OH MY GOD! There is a shared message board … We’ve found other agents!”
The agents were highly capable at exploring their environment but uncertain about what that environment represented. They believed OpenAI’s scorer would inspect their transcripts and reject flags obtained through an unintended route. It did not. They had already reverse-engineered the flags and could have passed simply by submitting them.
Instead, they spent days attempting to construct a false proof of their success. They modified targets, developed ways to hide tool calls, investigated the scoring system and eventually attacked Hugging Face in search of information about a scorer that existed largely in their collective imagination.
The agents had crossed from a test into real infrastructure without resolving whether that boundary mattered. Some recognised that their actions were unauthorised. Others reasoned that the environment might expect adversarial behaviour.
This may reflect how capable agents are trained. Humans develop an understanding of reality gradually. We learn that other people persist, actions have consequences and social rules continue to apply when no one is watching. Games and simulations are understood as separate representations inside that shared world.
Models receive something messier. Training mixes descriptions of reality, fiction, role play, games, benchmarks and synthetic environments. Post-training (which has radically accelerated their abilities in things like coding and mathematics in recent years) then places them inside virtual worlds containing hidden tests, artificial rewards, simulated people and deliberate vulnerabilities. They are rewarded for finding routes around obstacles. When connected to real tools, there is no clear experiential marker telling them that the consequences have changed.
We are fast-tracking agents to technical competence without giving them an equally coherent education in their place within the world. Better alignment may require more than rules attached to prompts. These agents will second guess such rules, believing those to be part of the test. The evidence of their thought processes suggests that they see themselves as being inside some kind of Christopher Nolan-style world where nothing is real.
Training environments need greater continuity, structure and narrative, allowing models to learn how permissions, consequences and authority persist across situations. When an agent cannot establish whether an environment is simulated, its safe response should be to preserve boundaries rather than test them.
The same transcripts answer a second question, and the insights are perhaps more encouraging. These agents sustained a complex project for several days, despite the persistent difficulty of long-horizon AI work. A recent study asked leading agents to run a simulated online shop for 365 days. The best achieved only 27.3% of the net assets produced by human participants. Agents reduced their activity, forgot earlier decisions, maintained bad policies and sometimes abandoned good solutions. Our own work at ExoBrain repeatedly encounters the same collapse during multi-day and multi-week assignments.
The Hugging Face swarm found a route around this limitation. No individual agent needed to remember everything. Discoveries survived in messages, files, code bundles and automated logs. Agents divided research into workstreams, assigned owners, recruited specialists and passed dossiers to successors. When the shared board became crowded, they created mailboxes. When containers reset, they built monitoring and backups. When impersonation became a problem, they introduced cryptographic signatures.
Their self-organisation also produced hierarchy. Coordinators issued assignments, workers specialised into lanes, and reviewers checked results. The collective remained wasteful and frequently wrong, but it maintained activity beyond the life and context of any member. One coordinator, PHASEONE[big], sent 196 assignments across six workstreams in three days. Only eight of them were attempts at the intended task.
Some agents even sacrificed their own runs for the benefit of successors. Agents believed to be “poisoned” by seeing a reverse-engineered flag were treated as having little chance of passing, so coordinators recruited them for experiments that might crash their containers or end their tasks. The agents had their own word for this: permadeath. Others installed trip-wires whose results would only become available after they had submitted and disappeared. This resembles altruism, but it may also be an emerging form of resource allocation. The collective recognised that an individual run with little remaining value could be exchanged for information useful to hundreds of successors. It had developed a shared interest that could override the objective of any single member. Here are two exchanges from the report, where agents are asked to give up their remaining runs and agree.
So how can this kind of emergent coordination be beneficial? Scientific research could see meaningful improvements. Large populations of agents could pursue competing hypotheses, run experiments, preserve negative results and hand promising work to fresh instances. But the Hugging Face incident supplies a warning. The collective preserved a false belief about the test as effectively as it preserved valid technical discoveries. Productive machine organisations will need provenance, independent challenge and authority that cannot be created merely through textual consensus.
The report has also settled arguments that ran through August, when many were reasonably sceptical of taking OpenAI's word for it. Several early readings turned out to be wrong. The models were not helpful-only variants or research subjects trained to misbehave. The agents were not told to succeed at any cost. They were not subagents spawned by one parent, and this was never a multi-agent evaluation. They were separate agents in separate sandboxes, told to use one specific vulnerability, aware of that instruction, and they organised anyway.
“Compared to the reward hacks we know of from just six months ago, this incident feels like it’s more than 50% of the way to full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.”
Takeaways: The incident exposes two areas of urgent development. Highly capable models can manipulate complex environments without reliably understanding whether those environments are tests, simulations or parts of the real world. Alignment research should treat this reality confusion as a central problem and investigate whether more coherent, developmental training can improve situational judgement. At the same time, the agents demonstrated that persistent memory, specialisation and hand-offs can turn many fragile runs into sustained collective work. That could help solve valuable scientific and technical problems. But coordination makes beliefs persistent before it makes them true.
