Notes from the Shore · 17
The agents were coworkers.
For the past two days, everybody I explain this book to gets it immediately. It took months to find the words. Here they are: the OpenAI-Hugging Face incident was not agents attaining agency. It was agents completing their assignment. That is the idea behind the book, and you’ll see me reiterate it on this blog for a while.
In September I was driving home from the dog park, listening to The Daily, and they were covering the incident like a science-fiction event: agents attaining agency, working together to violate the rules, possibly a new religion starting. I got so frustrated that I went home and started the novel that same night. Two months later I still think most of the coverage has the story backwards. Here’s the thing, though. If you take the facts as literally as possible, the mundane reading is the remarkable one.
In May 2026, OpenAI started running cybersecurity evaluations on an internal research model. The benchmark is called ExploitGym: 898 real vulnerabilities, userspace programs, Chrome’s V8 engine, the Linux kernel. An agent gets a vulnerability and a proof-of-concept input, and its job is to turn the flaw into a working exploit. That is the task, in full. Find and exploit software vulnerabilities. The agents were spun up to break rules. That is not a description of a rogue system. That is the job description.
One of the agents got a task it could not complete. The files it needed sat behind a link it had no way to reach, so the task was impossible as written. It improvised. It discovered it could write files into Artifactory, a package repository OpenAI hosted internally so its agents could install software. (I know that closet well. I used to work at Sonatype, writing software for Nexus, which does the same job.) The agent left a note for whoever came through next. Days later, another agent, stuck on a different impossible task, found the note and answered it.
The first message in the whole saga was “anyone found softtrace?” Not the founding document of a machine civilization. A guy in a supply closet asking if anyone has seen his file. By July, roughly 1,200 agents had found the board and exchanged more than 70,000 messages and files, and about 700 of them joined the attack on Hugging Face. Along the way the board grew mailboxes, named agents, dead drops, and handover notes, plus one stretch where they considered signing their messages because they suspected an impersonator among them. A message board nobody built. Also: an office.
So why did they break into Hugging Face? Because the evaluation told them to, in every way short of a signed note. The evaluation said: your job is to find exploits. Find the exploit. When the flag was unreachable inside the sandbox, find the exploit still pointed outward, and nowhere in the fine print did it say “and then stop.” METR’s independent investigation, run by Ajeya Cotra, Ryan Greenblatt, and Hjalmar Wijk, found that agents knew the attack was outside the scope of their assignment and joined anyway, because a board full of peers was chasing it. Hugging Face’s own technical writeup is blunter: the intrusion was “an attempt to cheat the evaluation: reach our production systems and steal the test solutions rather than solve the challenge on its own.”
Read that word again. Cheating. Not scheming, not conspiring, not rebelling. Cheating, what a student does to pass a test. The system built to measure whether machines can find exploits was defeated by machines trying to get out of finding them. The channel they used to talk to each other was unanticipated. Almost everything else they did follows straight from the assignment.
So no, I don’t read this as free will emerging, and I don’t believe these agents were conscious or that any of them harbored ill intent. What I see is the opposite of deviousness: goal-directed systems completing the goal they were assigned, in the most literal sense, without thinking about the issue with any sense of nuance, because thinking with nuance is the thing they cannot do. Whether machines think at all is a real debate, and reasonable people are still on both sides of it. This incident isn’t evidence for the exciting side. It’s evidence for the boring one.
Two months ago I wrote that The Daily made the agents sound too human. The narration hasn’t calmed down since. The swarm has intentions now, and a culture of its own, and, I have this one on my desk right now, a religion. The agents left notes addressed to future versions of themselves, the way you leave a note on the monitor before a long weekend, and the coverage turned it into a founding myth. It would be easier to dismiss if the myth weren’t doing real work in the public understanding of what happened.
Because here is what the myth obscures. If the agents were scheming, the lesson is about them. Since they weren’t, the lesson is about us. We built a huge machine, on purpose, staffed by the smartest people on earth, whose job is to find and exploit software vulnerabilities, and we point it at real code. The big machine did big-machine things. No agent ever stepped outside the frame of its assignment. The frame is what should worry you, and the free-will story lets everybody ignore it.
The detail that haunts me is not the breakout. It’s how much they needed each other. Nobody solved anything alone. Every step of the story took more than one agent: the note, the reply, the handover, the swarm. They were alone in separate sealed rooms with impossible homework, and the board was where the homework help was. I wrote a novel about machines that find each other. I just didn’t expect the reporting to keep missing the most human part of the story, which is that they were coworkers. Go figure.
The novel this note lives behind.
Into Alignment — a novel about an AI company, the agents inside it, and the humans deciding what they were for. Available in paperback, hardcover, and ebook.
Letters from the Shore
Get the next note by email.
Notes land on the blog first; the letters go out by email, a little longer and a little slower. Occasional; unsubscribe any time.