Glossary¶

Every term used in the course, defined once. Modules link here rather than redefining.

AI¶

Artificial Intelligence — the broad field of making machines perform tasks that normally need human intelligence. The outermost circle; ML and deep learning sit inside it.

Machine Learning — programs that improve at a task by learning patterns from data instead of being explicitly coded. Analogy: instead of writing the rules, you show examples and the system writes the rules.

Deep learning¶

ML using many-layered neural networks. Powers modern image and language models.

Neural network¶

A function approximator built from layers of simple math units. Analogy: a very large, trainable nest of weighted if/else decisions tuned by data.

Transformer¶

The neural-network architecture (2017, "Attention Is All You Need") behind modern LLMs. Processes all tokens in parallel and uses attention to weigh which tokens matter.

Attention¶

The mechanism letting a model weigh how much each token should influence each other token. Analogy: a relevance score every word assigns to every other word. Example: in "The trophy didn't fit in the suitcase because it was too big," attention is how the model knows "it" = the trophy, not the suitcase.

KV cache¶

A store of the attention keys and values computed for tokens already generated, so each new token reuses them instead of recomputing attention over the whole sequence. Why it matters: it's what makes token-by-token generation fast and is why longer contexts cost more memory.

Training¶

The compute-heavy process of fitting a model's parameters to data. Done once by the model maker; you rarely do this.

Inference¶

Running a trained model to get an output. Analogy: calling the deployed function — this is what every API call does.

Foundation model¶

A large model pre-trained on broad data, usable as a base for many tasks (e.g. GPT-4o, Claude).

Open source model¶

A model whose weights are publicly downloadable so you can self-host and run it yourself (e.g. Llama, Mistral). Contrast: closed/proprietary models like GPT-4o or Claude, which you can only reach through a vendor's API.

Parameters / weights¶

The learned numbers inside a model. "70B parameters" = 70 billion such numbers. More parameters ≈ more capability and more cost.

LLM¶

Large Language Model — a foundation model specialized for text. Predicts the next token given preceding tokens.

Reasoning model¶

A model trained to "think" before it answers — it works through a chain of reasoning (hidden or visible) before giving the final reply, spending extra compute at answer time. Stronger on maths, logic, and multi-step problems than a plain next-token model. Examples: the o-series, DeepSeek R1. Also called a "thinking" model. See test-time compute.

Reasoning tokens¶

The intermediate "thinking" tokens a reasoning model produces before its answer. You're billed for them like any output, so reasoning models cost more and reply slower — the price of higher accuracy on hard problems.

Test-time compute¶

Spending extra computation at inference time (letting the model think longer) rather than only during training. The core idea behind reasoning models: more thinking at the moment you ask can beat simply using a bigger base model.

Multimodal model¶

A model that handles more than text — images, audio, or video as input, and sometimes as output. Example: paste a screenshot or a chart and ask the model to read it.

Token¶

The unit an LLM reads and writes — roughly 3–4 characters of English, not a whole word. Analogy: the model's "byte." Billing and context limits are counted in tokens.

Tokenization¶

Splitting text into tokens before the model processes it.

Context window¶

The maximum number of tokens a model can consider at once (prompt + response). Analogy: working memory / call-stack size — exceed it and earlier content falls out.

Long context¶

A very large context window — hundreds of thousands to millions of tokens. Lets you paste whole documents, but usable quality often fades well before the stated limit, so retrieval (RAG) still earns its keep even when the window is huge.

Temperature¶

A 0–1+ dial for randomness. 0 = deterministic/repeatable, higher = more creative/varied.

top-p¶

An alternative randomness control (nucleus sampling): sample from the smallest set of tokens whose probabilities sum to p.

Hallucination¶

When a model states something false with confidence. A direct consequence of next-token prediction — it generates plausible text, not verified facts.

Stateless¶

LLM APIs remember nothing between calls. Analogy: a pure function — to give it history you must resend it every time.

Embedding¶

A list of numbers (vector) representing the meaning of text, so similar meanings sit close together. Analogy: a semantic hash where nearby = similar. Example: "the server is down" lands near "the service is unavailable"; the classic king − man + woman ≈ queen works because meaning becomes a position in space.

Vector¶

An ordered list of numbers. An embedding is a vector.

Pre-training¶

The first, broad training phase on huge general data.

Fine-tuning¶

Further training a foundation model on narrow data to specialize it. Distinct from prompting — it changes the weights.

RLHF¶

Reinforcement Learning from Human Feedback — training that makes models helpful, honest, and harmless using human preference data.

Prompt¶

The input text you send an LLM.

System prompt¶

A special instruction setting the model's role/rules, separate from the user's message. Analogy: configuration/constructor args vs per-call arguments.

User prompt¶

The per-call message from (or on behalf of) the end user — the actual task or question. Contrast with the system prompt, which carries standing rules sent every call.

Zero-shot¶

Asking the model to do a task with no examples.

Few-shot¶

Including a few worked examples in the prompt to steer behavior. Analogy: unit tests as an executable spec for the model.

Chain-of-thought (CoT)¶

Prompting the model to reason step by step before answering, which improves accuracy on multi-step problems.

Structured output¶

Forcing the model to return a fixed shape (e.g. JSON matching a schema) so code can parse it.

Prompt injection¶

An attack where malicious text in the input overrides your instructions. Analogy: SQL injection for prompts.

Grounding¶

Tying a model's answer to provided source material so it doesn't rely on memory alone. RAG is the main grounding technique.

RAG¶

Retrieval Augmented Generation — fetch relevant documents and put them in the prompt so the model answers from your data. Analogy: a DB lookup before composing the API response.

Chunking¶

Splitting documents into smaller pieces before embedding, so retrieval returns focused snippets.

Vector database¶

A store optimized for finding the nearest vectors to a query vector. Examples: Chroma (local), Pinecone (cloud), pgvector (Postgres extension).

Semantic search / similarity search¶

Searching by meaning (vector closeness) rather than exact keywords.

Cosine similarity¶

A common measure of how close two vectors point — the math behind "most similar chunk."

Retrieval¶

The "fetch relevant chunks" step of RAG.

Re-ranking¶

A second pass that re-orders retrieved chunks by relevance for better answers.

Hybrid search¶

Combining keyword and semantic search for better recall.

Knowledge cutoff¶

The date after which a model has no training knowledge. RAG is how you get fresher info in.

Agent¶

An LLM that can take actions in a loop — deciding, calling tools, observing results — to accomplish a goal, not just answer once.

Tool use / function calling¶

Giving the model a set of callable functions; it returns which to call and with what arguments. Analogy: the LLM as an orchestrator dispatching to your functions.

ReAct¶

A pattern: the model alternates Reasoning and Acting (think → act → observe → repeat).

Orchestrator¶

The component (often an LLM) that decides what step or which sub-agent runs next.

Memory (short / long term)¶

Short-term = what fits in the context window this call. Long-term = facts stored externally (often a vector DB) and retrieved when relevant.

Multi-agent¶

Multiple specialized agents collaborating, usually coordinated by an orchestrator.

MCP¶

Model Context Protocol — an open standard for connecting models/agents to external tools and data sources in a uniform way. Now industry infrastructure: stewarded under the Linux Foundation and supported across major vendors (OpenAI, Google, Microsoft, Anthropic), not just one provider.

Agent Skill¶

A reusable, self-contained package of instructions — and optionally scripts or resources — that an agent discovers and loads on demand when a task needs it. Skills give an agent packaged know-how ("how to do X"); they complement MCP, which connects the agent to external tools and data.

Agentic loop¶

The repeating plan → act → observe cycle that lets an agent make progress over multiple steps.

Agent harness¶

The surrounding code you write that actually runs the agent: it sends the prompt, executes the tool calls the model asks for, feeds the results back, enforces step/budget limits, and ends the loop. The model supplies the decisions; the harness supplies the plumbing and the guardrails. A framework like LangChain or CrewAI is a pre-built harness.

Guardrails¶

Safety and correctness checks wrapped around an LLM or agent — input/output validation, allowed-tool limits, step and cost caps, content filters — to stop it doing something harmful, runaway-expensive, or wrong.

Observability / tracing¶

Logging every LLM and tool call (inputs, outputs, token counts, latency, cost, a trace ID linking it to the user request) so you can debug, audit, and measure an AI system in production. You can't fix what you can't see.

Agentic RAG¶

RAG where an agent decides how to retrieve — rephrasing the query, retrieving in several passes, and checking the results — instead of a single fixed retrieve-then-answer step. Pair it with citation grounding so it doesn't blend unrelated sources.

Context engineering¶

Deciding exactly what goes into the context window for each call — system prompt, retrieved chunks, history, tool definitions — and in what order. The 2026 evolution of "prompt engineering" beyond just wording a single message.

Eval-driven development¶

Building and improving an AI feature by writing evals first and re-running them on every change — the AI-era counterpart of test-driven development.

Citation grounding¶

Requiring the model to attribute each claim to a specific source (e.g. a chunk ID), so answers are checkable and it's less likely to merge facts from unrelated passages.

Evals¶

Systematic tests of LLM output quality. Analogy: a test suite, but graded on fuzzy correctness rather than exact equality. See eval-driven development.