Home / Glossary
Glossary
The AI terms I use in my pieces, explained briefly. Where a piece goes deeper, the link is under the entry.
- agent
A program in which a language model generates, one step at a time, the actions to run (read a file, run a command, search the web) and gets the results back. The model generates the sequence of steps, one turn at a time, instead of code written in advance.
I explain it in “Do you know the difference between a workflow and an agent?”
- attention
The mechanism by which a model, while processing a token, computes how much weight to give each token in the context and combines them into a weighted sum. The name is a metaphor: it is a calculation of weights, not attention in the human sense.
- auto-compaction
What the harness does when the context goes over a token threshold. It replaces the conversation history with a summary that the model generates. From then on, the model receives the summary, not the original messages.
I explain it in “Auto-compaction is a colossal piece of crap”
- CLAUDE.md
The plain-text instructions file that Claude Code loads at the start of every session, with the rules and project information you would otherwise have to repeat. It is context, not an enforced rule: a line in CLAUDE.md does not force an outcome, it shifts a probability.
I explain it in “Claude Code hygiene”
- compaction
Replacing the older part of a conversation with a summary the model generates, to free up space in the context window. It is not lossless compression but a rewrite: what does not make it into the summary no longer exists for the model.
I explain it in “Auto-compaction is a colossal piece of crap”
- context rot
The drop in accuracy with which a model retrieves and uses information as the context grows longer. The more tokens in the window, the less reliable the answers become, even on simple tasks.
I explain it in “Claude Code hygiene”
- context window
The maximum number of tokens that fit in one request, including the model’s reply. It holds everything: the instructions, the conversation, the files it has read, and the tool outputs.
I explain it in “Auto-compaction is a colossal piece of crap”
- embedding
A list of numbers that represents the meaning of a piece of text. Texts with similar meanings get similar numbers, so you can search by similarity of meaning and not only by identical words.
- harness
The program built around the model. It rebuilds the context and sends it with every request, and runs the tools the model asks to use. Claude Code, Codex, and Cursor are all harnesses.
I explain it in “Do you know the difference between a workflow and an agent?”
- inference
Using an already trained model to generate a response to a prompt, one token at a time. It is distinct from training, the phase in which the model is built from data.
- J-space
A small set of internal activity patterns in Claude (a few dozen at a time, less than a tenth of its total activity), found by Anthropic with a technique called the J-lens (the J comes from the Jacobian, a mathematical concept). Each pattern is tied to a word; Anthropic compares the set to the “global workspace” of neuroscience, but states that the work does not settle whether Claude is conscious.
- logit
The raw score the model computes for each possible next token, before it is turned into a probability. The token with the highest logit is also the most probable, but which one gets generated depends on how the output is sampled (temperature, top-k).
- MCP (Model Context Protocol)
An open standard, introduced by Anthropic in November 2024, that lets an AI application connect to external tools and data sources always in the same way (the official image is the USB-C port). An MCP server is the small program that, for a service (a calendar, an image generator), exposes those tools to the application.
I explain it in “Frontend is Dead”
- prompt
The text you give a language model as input to get a response: the question, the instructions, the examples. The model generates a plausible continuation of that text, so changing the words of the prompt changes the response.
- prompt caching
The provider keeps the start of the context, already processed, for a while. If the next request starts with exactly the same tokens, that part is read from the cache instead of being processed again, and it costs much less.
I explain it in “Auto-compaction is a colossal piece of crap”
- RAG (retrieval-augmented generation)
Before answering, a system searches a collection of documents for the passages most relevant to the question and adds them to the prompt. It works like an open-book exam: the model generates its answer from those texts, and not only from the data it was trained on.
- skill
A folder with an instructions file (SKILL.md) that teaches Claude Code a task. Only the name and description stay in the context at all times; the rest loads only when the skill is used, by Claude or by you with /name.
I explain it in “Claude Code hygiene”
- stateless
Keeping nothing from one request to the next: on every turn the model receives the whole context from scratch. It is like having to reread the entire apartment-building group chat every time you reply “ok.”
I explain it in “Auto-compaction is a colossal piece of crap”
- subagent
A second agent that the main session launches with a specific task; it works in a context of its own and returns a report at the end. Only the report goes back to the main session, so long reads don’t fill its context.
I explain it in “Do you know the difference between a workflow and an agent?”
- temperature
A parameter that controls how random a model’s choice of the next token is: at low temperature it almost always picks the most probable tokens, at high temperature it gives more room to less probable ones. Even at temperature 0, the responses are not guaranteed to be identical.
- token
A piece of text that a language model receives and generates: a short word, part of a word, or a punctuation mark. It is also the unit that requests are billed in.
I explain it in “Auto-compaction is a colossal piece of crap”
- tool use
A model’s ability to generate a request to use a tool (a function someone wrote, like “search the web” or “read this file”), with its parameters. It is not the model that runs it but the program around it, which then sends the result back.
I explain it in “Do you know the difference between a workflow and an agent?”
- top-k
A way of choosing the next token, in which the model draws at random among the k most probable tokens (for example the top 10) and ignores all the others. It is used to rule out very unlikely choices.
- transformer
The neural network architecture introduced in 2017 in the paper “Attention Is All You Need” and used by most current language models. It processes text with the attention mechanism alone, without the recurrent networks that stepped through it one position at a time, so it trains much faster and in parallel.
- workflow
A system in which you write the sequence of steps yourself, in code, before you start: the model carries out the individual steps but does not decide the route. It is a train: it only goes where the tracks are already laid.
I explain it in “Do you know the difference between a workflow and an agent?”
I’ll post the next piece on Telegram
Keep up with AI without chasing everything: you’ll see what changed, what to try right away, and what to update before it becomes a problem. The channel is in Italian.
Notify me on Telegram