A lot of training,
a little bit of math, and some chance.

I spent part of this week doing something writers are probably not supposed to do: running analysis on my own novel. I went looking for the words that keep returning, chapter by chapter, and charted where each theme surfaces and dips. That analysis turned into The Tides, a page on this site where you can follow a current — memory, rules, care, doubt — across all sixteen waves of Into Alignment and touch any crest to see the language underneath. It started as a pile of graphs. It ended up being one of my favorite things I’ve made for this book.

Here’s the thing worth saying out loud, though: the novel is about AI, and the novel is not a technical document. So I put together something else to travel with it — a research companion, called AI & Autonomy, that tries to explain some of the bigger topics underneath the story. How a language model learns. What memory does to an agent. Why an evaluation can mislead you. I’ll admit this may be a slightly presumptuous thing to publish alongside a novel — it’s not my first novel, but you take my point. I just wanted to do something interesting, and the questions the book sits on top of deserve real answers somewhere nearby.

The same curiosity has pulled me somewhere new this week: open-weight models, and poetry. I’ve started wondering what we could learn by looking at how the most studied texts in the world are actually represented inside these things. Shakespeare’s sonnets are the obvious candidate — other than the Bible, probably the most studied text there is, so there’s plenty of existing scholarship to lean on. I’m not the first person to study this mathematically, by a long shot. What I want to do is measure how the sonnets are represented across several open-weight models: Qwen, the popular one from the Chinese provider; whatever Meta shipped last week; and Gemma, Google’s. The models differ in training, in size, in language. Do they hold a sonnet the same way?

The more you read about how these models actually work, the more fascinating the whole thing gets. I’m a technical person, but one thing I’m trying to do with this project is explain the machinery for people who don’t have a CS or EE degree, because most people use these tools every day without ever stopping to look at what’s happening behind the scenes. The short version: when a model reads a text, it first cuts the text into tokens, and then it calculates vectors — activation vectors — that carry what the text means at each stage of being processed. What I’m building captures what a text looks like at different phases as it moves through each model, so the representations can be compared. I keep reaching for a clean way to say that and not quite finding it — the models don’t “represent” a poem the way a filing cabinet holds a folder, and the honest description is messier than the metaphor. I’ll blog the process as I go and try to get the language right in public, where getting it wrong is cheaper.

Because that’s the part people miss. There is no little reader in there, no ghost following the argument of a sonnet. A lot of training, a little bit of math, and some chance. It’s deeper than that. The math is the interesting part, and I’m not sure I’m able to explain it so most people can understand. I’ll keep trying in these notes. But here’s what I’m sure of: even if you think these LLMs understand what you’re asking them to do, they don’t. And sometimes, talking to humans, I’m not sure they understand either.