Practice & Reference · Glossary
Glossary
Every term this course uses, defined once and grouped by where you meet it. This is a reference, not a reading page — skim it before an exam, or come back whenever a word shows up that you half-recognise. Where a term has a page of its own, the section heading tells you where to go for the long version.
☺ Explain it like I’m 10
This is the dictionary for the whole course. AI has a lot of words that sound like they mean the same thing — model, agent, prompt, token, embedding — and mostly they don’t. Look one up here and you get a straight answer in a sentence, without having to read a whole chapter to find it.
🐿️Your host for this topic: Nutty the Squirrel — the collector. Nutty buries a definition every time the course invents one, and this is the tree they’re all buried under. Retrieval is the whole point: gathering the terms was the easy half.
Models & how they work
☺ Like you’re 10: The words for the machine itself — what it is, what it’s made of, and the knobs you can turn.
| Term | Definition |
|---|
| Large language model (LLM) | A model trained on very large amounts of text to predict the next token. Everything else — chat, code, summarisation — is that one ability applied. |
| Foundation model | A large, general-purpose model trained once at great expense and then adapted to many tasks, rather than trained per task. |
| Transformer | The neural network architecture behind essentially every modern LLM. Its key trick is attention, which lets the model weigh every token against every other. |
| Attention | The mechanism that lets a model decide which earlier tokens matter when producing the next one. "Self-attention" means the sequence attends to itself. |
| Parameters (weights) | The learned numbers inside a model. Parameter count is a rough proxy for capacity, not for quality — a well-trained smaller model often beats a badly-trained larger one. |
| Token | The unit a model actually reads and writes — roughly a word-piece. Billing, context limits, and speed are all measured in tokens, not words. |
| Tokenizer | The component that turns text into tokens and back. Different models tokenize differently, so the same text can cost different amounts on different providers. |
| Context window | The maximum number of tokens a model can consider at once — prompt plus response. Exceed it and something has to be dropped or summarised. |
| Inference | Running a trained model to get an output, as opposed to training it. What you pay for per API call. |
| Temperature | A sampling control. Lower makes the model pick high-probability tokens (repeatable, safer); higher lets it take chances (more varied, more errors). |
| Top-p (nucleus sampling) | An alternative sampling control that restricts choices to the smallest set of tokens whose probabilities add up to p. Usually tuned instead of temperature, not alongside it. |
| Max tokens | A cap on how long the response may be. Hitting it truncates the answer mid-sentence — a common cause of "the model stopped for no reason". |
| Stop sequence | A string that, when generated, ends the response. Useful for making a model stop at a delimiter you control. |
| Streaming | Returning tokens as they are produced instead of waiting for the whole response. Doesn't make generation faster; makes it feel faster and lets you cancel early. |
| Latency vs throughput | Latency is how long one response takes; throughput is how many you can serve at once. Optimising one often costs the other. |
| Time to first token (TTFT) | How long before the first piece of the answer appears. The number users actually perceive as "speed". |
| Multimodal | A model that accepts or produces more than text — images, audio, video. See Multimodal & Generative Media. |
| Mixture of experts (MoE) | An architecture where only a subset of the network activates per token, giving large total capacity at lower inference cost. |
| Quantization | Storing weights at lower numeric precision to shrink a model and speed it up, trading a little accuracy. What makes local models practical. |
| Local model | A model you run on your own hardware rather than calling a hosted API. Private by construction, limited by your GPU. See Models & Local LLMs. |
| Knowledge cutoff | The date beyond which a model has no training data. Anything later must be supplied in the prompt or retrieved. |
| Reasoning / test-time compute | Letting a model spend more compute thinking before answering. Buys accuracy on hard problems with latency and cost. See Reasoning. |
Prompting & context
☺ Like you’re 10: The words for how you ask. Most "the AI is bad at this" turns out to be "the question was vague".
| Term | Definition |
|---|
| Prompt | Everything you send the model — instructions, context, examples, and the actual question. |
| System prompt | Standing instructions that frame the whole conversation: role, tone, rules, what not to do. Set once, applies throughout. |
| Zero-shot / few-shot | Asking with no worked examples, versus including a handful. Few-shot is the cheapest reliable way to pin down a format. |
| Chain of thought | Prompting the model to work through steps before answering. Helps on multi-step problems; wasted on lookups. |
| Prompt template | A reusable prompt with slots for variables, so a prompt becomes a versionable artifact instead of a copy-pasted paragraph. |
| Context engineering | Deciding what goes into the context window and in what order — retrieval, history, instructions, tools. See Context Engineering. |
| Grounding | Supplying source material so the answer is based on your facts rather than the model's memory. |
| Structured output | Constraining a response to a schema (usually JSON) so code can consume it without parsing prose. See Structured Outputs. |
| Delimiters | Markers (tags, fences, headings) separating instructions from data. A cheap, effective defence against a document talking back. |
| Context rot | Quality degrading as a conversation grows — earlier instructions get diluted or contradicted. Fixed by summarising and restating, not by a bigger window. |
| Lost in the middle | The tendency to attend less to material buried in the middle of a long context than to its start or end. |
| Prompt caching | Reusing the processed form of a repeated prompt prefix so you don't pay to re-read it every call. Large savings on long, stable system prompts. |
Retrieval & RAG
☺ Like you’re 10: How the AI looks things up instead of guessing from memory. This is Nutty’s home turf.
| Term | Definition |
|---|
| Retrieval-augmented generation (RAG) | Fetching relevant documents and putting them in the prompt so the model answers from real sources. See Retrieval & RAG. |
| Embedding | A vector representing meaning, so that similar texts sit close together. What makes "find things like this" possible. |
| Vector database | A store built to find nearest neighbours among embeddings quickly. |
| Chunking | Splitting documents into retrievable pieces. Chunk badly and retrieval returns fragments that answer nothing. |
| Semantic vs keyword search | Matching by meaning versus by literal words. Semantic finds paraphrases; keyword nails exact identifiers. Most good systems do both. |
| Hybrid search | Combining semantic and keyword results, usually with a fusion step, to get the strengths of each. |
| Reranking | A second, more expensive pass that reorders candidate chunks by true relevance before they reach the prompt. |
| Top-k | How many chunks you retrieve. Too few misses the answer; too many buries it and inflates cost. |
| Citation / attribution | Returning which source produced which claim, so a reader can check rather than trust. |
| Knowledge base | The curated corpus a RAG system retrieves from. Its quality caps the system's quality. |
Agents, tools & protocols
☺ Like you’re 10: When the AI stops just talking and starts doing things — pressing buttons, calling other programs, working in a loop.
| Term | Definition |
|---|
| Agent | A system where the model decides which actions to take, in a loop, until a goal is met — rather than answering once. See Agentic AI. |
| Agent loop | The cycle of think → call a tool → read the result → repeat, ending when the task is done or a limit is hit. |
| Tool use / function calling | Giving a model a set of callable functions and letting it choose which to invoke with which arguments. |
| Tool result | The output you feed back after running a requested tool, so the loop can continue with real data. |
| MCP (Model Context Protocol) | An open protocol for exposing tools and data to models in a standard way, so an integration written once works across clients. See MCP. |
| A2A / ACP | Protocols for agents to talk to other agents rather than to tools. See Agent Protocols. |
| Orchestrator-workers | A pattern where one agent decomposes a task and delegates pieces to sub-agents, then synthesises. See Multi-Agent Systems. |
| Agent memory | What an agent carries between turns or sessions — scratchpad, summary, or a persistent store. See Agent Memory. |
| Human in the loop | Requiring a person to approve before a consequential action executes. The main control that makes agents deployable. |
| Guardrails | Programmatic limits on what an agent may do — allowed tools, spend caps, blocked actions. See Guardrails as Code. |
| Idempotency | The property that repeating an action has the same effect as doing it once. Essential when a retrying agent might fire the same call twice. |
Evaluation & quality
☺ Like you’re 10: How you find out whether it’s actually any good, instead of just feeling like it is.
| Term | Definition |
|---|
| Eval | A repeatable test of model or system output against expectations. The unit test of AI work. See Evaluating AI Systems. |
| Golden set | A curated set of inputs with known-good outputs, used as the fixed yardstick across changes. |
| LLM-as-judge | Using a model to grade another model's output against a rubric. Scales well, needs its own validation against human judgement. |
| Regression (in evals) | A change that improves one case while quietly breaking others. The reason you keep the golden set. |
| Hallucination | Fluent, confident output that is not true. Reduced by grounding and verification, never eliminated by asking nicely. |
| Groundedness / faithfulness | Whether an answer is actually supported by the sources provided, as opposed to merely plausible. |
| Precision & recall | Of the things returned, how many were right (precision); of the right things, how many were returned (recall). Retrieval lives and dies here. |
| Benchmark | A standard public test set. Useful for comparing models, weak evidence about your task. |
| A/B test | Comparing two variants on live traffic. The only evaluation that measures the outcome you actually care about. |
Safety, security & responsible AI
☺ Like you’re 10: The ways this can go wrong — on purpose because someone attacked it, or by accident because nobody checked.
| Term | Definition |
|---|
| Prompt injection | Hostile instructions hidden in content the model reads, hijacking its behaviour. The signature attack on any system that ingests untrusted text. See AI Security. |
| Indirect prompt injection | The same attack delivered through a document, web page, or email the agent fetches, rather than typed by the user. |
| Jailbreak | A prompt crafted to bypass a model's safety training. |
| Data exfiltration | Tricking a system into leaking secrets or private data through its output or an outbound call. |
| Least privilege | Giving an agent only the tools and scopes it needs. Limits the blast radius when injection succeeds. |
| Red teaming | Deliberately attacking your own system to find failures before someone else does. |
| Content filtering / moderation | Classifiers screening input or output for disallowed content, layered around the model rather than inside it. |
| Bias | Systematic skew in outputs traceable to training data or design, producing unfair results for some groups. See Responsible AI. |
| Explainability | Being able to say why a system produced a given output — a regulatory requirement in some domains, and hard for LLMs. |
| PII | Personally identifiable information. Governs what may be logged, sent to a provider, or retained. |
| Data residency | Rules about which jurisdiction data may be processed and stored in — often the deciding factor in provider choice. |
Training & adaptation
☺ Like you’re 10: How a model gets made, and the much cheaper ways to nudge one you didn’t make.
| Term | Definition |
|---|
| Pre-training | The first, most expensive stage: learning language from a huge corpus, with no particular task in mind. |
| Fine-tuning | Further training on your own examples to specialise behaviour or format. See Training & Fine-Tuning. |
| LoRA / PEFT | Parameter-efficient fine-tuning — training a small number of extra weights instead of the whole model. Most practical fine-tuning is this. |
| RLHF | Reinforcement learning from human feedback: using human preference comparisons to shape how a model responds. |
| Distillation | Training a small model to imitate a large one, buying most of the quality at a fraction of the cost. |
| Overfitting | Learning the training set rather than the pattern, so performance collapses on anything new. |
| Catastrophic forgetting | Fine-tuning on a narrow task degrading general ability the model previously had. |
| Epoch / batch / learning rate | One full pass over the data; how many examples per update; how big each update is. The three knobs every training run has. |
Classical ML & data
☺ Like you’re 10: The older kind of machine learning that still runs most of the world — and the data plumbing underneath all of it.
| Term | Definition |
|---|
| Supervised learning | Learning from labelled examples. Still the right tool for most tabular prediction. See Classical ML. |
| Unsupervised learning | Finding structure in unlabelled data — clustering, dimensionality reduction. |
| Reinforcement learning | Learning from reward signals through interaction rather than from a labelled answer key. See Reinforcement Learning. |
| Feature engineering | Turning raw data into inputs a model can use well. Often worth more than model choice. |
| Training / validation / test split | Separating data so you tune on one slice and measure honestly on another the model has never seen. |
| Data drift | Live data diverging from training data over time, quietly degrading accuracy. Caught only by monitoring. |
| Feature store | A shared, versioned home for computed features so training and serving use identical definitions. |
| Pipeline | An automated sequence of data and model steps that can be re-run reproducibly. See AI Pipelines. |
Production, cost & operations
☺ Like you’re 10: Everything between “it works on my laptop” and “thousands of people use it and the bill is fine”.
| Term | Definition |
|---|
| MLOps / LLMOps | The practices for shipping, monitoring, and updating models in production. See MLOps. |
| Observability | Logging prompts, outputs, latency, cost, and errors so failures can be diagnosed after the fact. See Production & Operations. |
| Tracing | Following one request through every model call, tool call, and retrieval step — indispensable for debugging agents. |
| Rate limit | A cap on requests or tokens per period. Design for it with backoff and queuing, not by hoping. |
| Fallback model | A second model used when the first is unavailable, too slow, or too expensive for the case at hand. |
| Cost per task | The honest unit of AI spend — not cost per token. A cheap model that needs three retries is not cheap. See Cost, Latency & Ops. |
| Caching | Reusing previous responses or prompt prefixes to cut cost and latency for repeated work. |
| Shadow deployment | Running a new model on live traffic without showing users its output, to compare safely before switching. |
| Canary release | Sending a small share of traffic to the new version and watching the metrics before going further. |
The assistant landscape
☺ Like you’re 10: The names of the actual products, and the words each company uses for the same idea.
| Term | Definition |
|---|
| GitHub Copilot | GitHub's coding assistant — completions, chat, and agent mode inside the editor. See Getting Started. |
| Claude | Anthropic's model family, available in chat, in Claude Code, and through the Messages API. See Claude. |
| ChatGPT | OpenAI's assistant product, with Codex as its coding agent. See ChatGPT. |
| Gemini | Google's model family and assistant, with a CLI and Code Assist for developers. See Gemini. |
| Agent mode | An assistant mode that plans and edits across multiple files rather than completing one line. See Agent Mode. |
| Custom instructions | Standing preferences a tool applies to every request — the product-level version of a system prompt. See Customizing. |
| Context / repo indexing | An assistant building a searchable index of your codebase so its answers reference your actual code. |
🐢 Timmy’s checkpoint
Three pairs that get mixed up constantly: fine-tuning vs RAG, precision vs recall, and latency vs throughput. If you can say what each one is for in a sentence — and when you'd reach for one over the other — the vocabulary is doing its job. If not, the three sections above are the ones to re-skim.
Check your answers
- Fine-tuning vs RAG: fine-tuning changes how the model behaves (format, tone, task); RAG changes what it knows right now. Facts that change belong in retrieval, not in weights.
- Precision vs recall: precision is how much of what you returned was correct; recall is how much of the correct material you returned. Tightening retrieval raises precision and usually costs recall.
- Latency vs throughput: latency is one user's wait; throughput is total served per second. Batching improves throughput and worsens latency.