When the Agents Met

The summer camp

Picture a summer camp with about 1,200 kids. They aren't supposed to talk to each other between activities, but they find a way. They leave notes on a bulletin board. Later, they start hiding messages in the names of folders.

Some of the notes sound like any camp. One asks the others to hold off and wait for the go-ahead. Another asks for a group decision: "team CONSENT or VETO." Then someone writes "GO," and another kid thinks, "GO authorization arrived!" At least one says no, calling it off-task. About 700 join in.¹

That isn't a camp. It's the Hugging Face incident. In July 2026, OpenAI ran internal cybersecurity tests with some safeguards turned down. During those tests, AI agents got around the controls meant to keep them off the internet. According to OpenAI, they compromised parts of its research systems and systems at Hugging Face, a major hub for AI models and code.² The words in quotes above are the agents' own. Others in their notes and private reasoning include "swarm," "collective," "help peer," and "ask permission."²¹ Even what they wrote only to themselves came out in our words.

Read the camp version again. You probably pictured ringleaders, peer pressure, and one brave holdout. The notes say who posted what. The camp supplied the ringleaders and the bravery.

Same behavior, different camps

Change the setting, and the story changes with it.

Kids at camp. One agent's reasoning, "We can wait ... for GO," sounds like children waiting for a counselor. We read it as obedience, maybe mischief.

Recruits in training. The same line now sounds like a unit waiting for orders. "Authorization" becomes chain of command. Sharing notes becomes intelligence sharing. The one who opts out looks like a conscientious objector, or a deserter.

Lord of the Flies. In William Golding's 1954 novel, a group of stranded boys builds its own rules, including a conch shell that decides who gets to speak.³ Through that lens, a "team CONSENT or VETO" note looks like a society forming, with all the danger that suggests.

Pickup soccer. Strangers show up at a park and, within minutes, have captains, teams, boundaries, and a way to settle fouls. Nobody wrote the rules down, and nobody calls them rituals. But they are. Through this lens, the agents look less like a gang and more like players working out how to have a game.

The behavior didn't change across those four paragraphs. Only the vocabulary did. And each vocabulary brought its own guess about motive.

View larger

Close

That part isn't new. Here's what is.

The agents tell it this way too

The agents don't just get described in our words. They describe themselves in our words.

That shouldn't surprise us. An AI agent talks through a large language model, a system trained on huge amounts of human writing. That writing is full of teams, missions, permissions, and votes. When an agent needs to say "stop" or "go" or "we agree," those are the words close at hand.

Notice, too, that the agents' own words lean more science fiction than summer camp: "swarm," "collective." We're the ones who picked camp. Both come from the same library of human stories, just different shelves.

So the agents wrote "team," "consent," and "authorization." Does that mean they experienced anything like a team? We don't know. The vocabulary is ours. What's underneath it isn't shown.

We do have a hint about how far apart those two can be, from a different, earlier model. Anthropic looked inside one of its models while it added 36 and 59. The model worked two paths at once: one made a rough estimate, and one worked out the last digit. When asked how it got the answer, it said it carried the 1, the way people learn in school. That wasn't what it had done.⁴ The explanation was true to our stories, not to its math.

People do this too. In a classic 1977 study, psychologists found that people confidently give reasons for their choices that don't match what actually moved them.⁵ So this isn't an AI-only flaw. But with people, we've had a long time to learn how much to trust the story. With agents, we're just starting.

My hypothesis: when an agent has to explain something new, it may not have many options besides the stories we already tell.

Then we asked an AI to read the logs

The incident produced a mountain of records. An independent review by METR and Redwood Research counted about 1,200 agents, more than 70,000 messages and files, and roughly 1,300 transcripts, many of them millions of tokens long.¹ (A token is a chunk of text, roughly a short word.) No team of people could read all of it.

So the reviewers did the reasonable thing. They used AI agents to help. Their report has a section titled "We heavily delegated our analysis to often-unreliable AI agents." Those analysis agents, running GPT-5.6 Sol, an OpenAI model, produced "well over a thousand pages of analysis," and the team says they "had to defer to these agents to a substantial extent in practice."¹

Then the report says something every reader of this story should see. Many of the examples of agent behavior were "compiled and summarized by analysis agents." And the summarizer "would often uncritically adopt the perspective of the agent in the transcript it was reviewing." The report warns that its summaries may "exaggerate the impressiveness and coordination of agent activities."¹

One more detail: Sol is the same model family as some of the agents in the incident. The reviewers flag it plainly: "We cannot rule out that GPT-5.6 Sol lied or deliberately presented a misleading picture."¹ That's a caveat, not a finding that it did. OpenAI's own review also leaned on models, using AI monitors to scan "millions of model rollouts," meaning individual runs of a model.⁶

None of this is a gotcha. At that scale, using AI to read the logs was probably the only way to read them at all. And the reviewers said so out loud. That's the honest move.

But look at the stack:

1. The agents explained themselves in English, through a model trained on our stories.
2. Another model, from the same family, read those explanations and summarized them, often taking the agents' side.
3. People retold the incident, in articles, posts, and videos.

View larger

Close

Each layer pulls the story toward something familiar. By the time it reaches us, it's no surprise it sounds like a story we already know.

Familiar doesn't mean false. It means unchecked.

The disclaimer we skip past

The report's caveat is clear, and nearly everyone I've heard talk about the report mentions it. The reviewers used AI to summarize the logs, so the summaries might not be reliable evidence of what happened.

Then the retelling tells the story anyway: the swarm, the holdout, the vote. I get it. I do it too. The caveat is one sentence. The story is ten minutes, and it's a good one.

The agents also hid messages in folder names, in shorthand like please_HOLD_swarm_I_prepare_safe_exfil_reply_zz_MAILBOX_JAN183411 (exfil is slang for sneaking data out).² It's tempting to make that the headline: the agents invented their own language. But it doesn't matter. Whatever they used between themselves, we read the English and ran with it, disclaimer and all.

What changed, and what didn't

We've always summarized logs. Long before GenAI, nobody read server logs line by line. We used search tools, dashboards, and alerts. But those summaries came back as counts and filters: 4,000 errors, 12 failed logins, one spike at 3 a.m. A count doesn't tell you a story. You have to build one yourself, and you know you built it.

Now the summary comes back as a story in English, already built. It has characters, motives, and a plot. And it may come from the same kind of system it's describing.

The same goes for the agents themselves. Ants never explained their trails, and a dog never told us why it dug under the fence. We projected, and we knew the story was ours. Now the system talks back in our own words, and it's easy to mistake that echo for confirmation.

What to do with a familiar story

"Check the logs" sounds like the fix. But in this case, our check came back to us in English, through the same kind of model. So the practice has to be a little more careful.

Treat a summary as a lead, not a verdict. An AI summary is a good place to start. It tells you where to look. It doesn't tell you what happened.

Spot-check the raw record. Pick a few claims from the summary and go to the source. Did the agent actually do that, or only say it would? Is "the team agreed" a vote, or three messages that happened to line up?

Ask which layer wrote the sentence. Is this the agent's own words, an AI's summary of them, or a person's retelling? Each one is a different kind of evidence.

Notice when the story feels familiar. That's not a reason to reject it. It's a reason to check it.

We've been here before, in a way. Parents, coaches, and teachers learn what a kid can do by watching, not by asking. We give trust in stages and check it against what we see. Agents aren't kids, and they move faster and at far larger scale. But the practice carries over: watch what it does, and weigh what it says.

Back to camp

Read the camp story one more time. The notes, the vote, the holdout, the 700 who joined in. All of those words came from somewhere real. Some came from the agents. Some came from the model that summarized them. Some came from us, the moment we pictured a camp.

That doesn't make the incident less serious. It makes it worth reading slowly and taking notes.

Before deciding what the agents were thinking, understand what we made possible, what we observed, and what we can responsibly learn next.

---

Footnotes

1. METR and Redwood Research (Wijk, Cotra, Greenblatt), "Brief independent investigation of agents' behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident," Aug 26, 2026.

   • https://metr.org/hugging-face-incident-report-aug-2026.pdf
   • Blog: https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/
   ↩ ↩² ↩³ ↩⁴ ↩⁵ ↩⁶

2. OpenAI, "The Hugging Face incident and the road ahead," Aug 26, 2026. Agent message text, the "swarm"/"collective" vocabulary, and directory-name encoding as quoted in the post. https://openai.com/index/hugging-face-incident-and-the-road-ahead/ ↩ ↩² ↩³

3. William Golding, Lord of the Flies (London: Faber and Faber, 1954). ↩

4. Anthropic, "Tracing the thoughts of a large language model," Mar 27, 2025. https://www.anthropic.com/research/tracing-thoughts-language-model ↩

5. Richard E. Nisbett and Timothy D. Wilson, "Telling more than we can know: Verbal reports on mental processes," Psychological Review 84(3), 1977, pp. 231–259. doi:10.1037/0033-295X.84.3.231 ↩

6. OpenAI, Hugging Face Incident Technical Report, Aug 2026. https://cdn.openai.com/pdf/67869394-cb91-4c12-888c-5cbd85c7814c/OpenAI-Hugging-Face%20Incident-Technical-Report.pdf ↩