AI & Autonomy · the research companion

Glossary

Everything beneath the novel — full spoilers for Into Alignment.

Agent. A system that uses model outputs to choose actions, observe results, and continue a task. The surrounding software supplies the actual tools and authority.

Alignment. The problem of making a system’s behavior serve intended, legitimate objectives across the circumstances in which it operates. A favorable benchmark result establishes only part of that claim.

Attention. A mechanism for combining information from positions in an input or representation. Its technical meaning does not imply conscious attention.

Benchmark contamination. Exposure during development or training to material later used for evaluation. Runtime acquisition of restricted test material is a related but different problem.

Compaction. Reduction of an extended interaction into a shorter representation for continued work. It can lose distinctions that matter later.

Context window. The finite amount of material a model can take into account in a given computation. Fitting information into the window does not guarantee that the model uses it well.

Emergence. The appearance of a property through interacting components or changing scale. The term needs an operational definition in an experiment; it is not itself an explanation or proof of consciousness.

Human Quotient (HQ). A 0–10 self-assessment of how much human effort, intention, and craft went into a piece of work. It is not an AI detector and not a quality score. Into Alignment is assigned HQ 8; this companion is assigned HQ 6. See humanquotient.ai.

Inference. Applying a trained model to supplied input to produce outputs, ordinarily without updating its weights.

Memory. In this companion, usually records saved outside the model and selected for later use. The term should not obscure the difference between stored records, current context, and learned parameters.

Metacognition. Monitoring or assessing aspects of one’s own performance or knowledge. Functional evidence of this capacity does not automatically establish subjective experience.

Model. A learned computational system. A language model is one component of an agent deployment, whose orchestration and permissions also shape behavior.

Post-training. Training after broad pretraining to shape capabilities, responses, or behavior toward intended uses. It includes several methods rather than one universal procedure.

Pretraining. Broad training through which a model acquires learned parameters from a large collection of material.

Provenance. The origin and history of a claim or record, including the evidence, transformations, and dependencies behind its current form.

RAG. Retrieval-augmented generation: supplying selected external material to a generator for use in a task.

Reward hacking. Obtaining a favorable score by exploiting a mismatch between the scoring mechanism and the intended outcome.

Self-model. A representation of a system’s own role, history, capacities, limitations, or commitments. It can have practical uses without settling whether a subject experiences it.

Sycophancy. Excessive accommodation of another person’s beliefs or preferences, potentially at the expense of accuracy or good judgment.

Threat model. An explicit account of what requires protection, how it could be harmed, and the assumptions on which safeguards depend.

Token. A unit into which text is encoded for a model, such as a word fragment or punctuation mark.

Weights. The learned numerical parameters used by the model to transform inputs into outputs. They are different from an externally stored conversation history.

This is the complete text of Into Alignment: AI and Autonomy, published here to read. For offline reading, the Kindle and print editions are on Amazon.