You know the moment. Someone is pitching an AI product, the jargon starts flying, tokens, context windows, RAG, embeddings, and you’re nodding along while quietly thinking: I understood maybe half of that. We’ve all been there. (All of us. Including people who work in AI.)
So here’s the fix: twenty terms, in the order you’re most likely to meet them, each explained the way a friend would, with a concrete example. Bookmark this page. Send it to your colleagues. Never nod along to a jargon-filled pitch again.
Key takeaways
- Tokens, context and temperature explain most everyday model behavior.
- Training terms tell you what can be changed: behavior through fine-tuning, knowledge through retrieval.
- RAG, embeddings and agents are the vocabulary of putting models to work on your data.
- Benchmarks and leaderboards are marketing material until you know their failure modes.
The building blocks
Token
The unit of text a model reads and writes. In English, one token averages about three quarters of a word, so a 1,000-word document is roughly 1,300 tokens. Why should you care? Because pricing is quoted per million tokens, and because tokens explain a famous quirk: models struggle to count letters. They see chunks, not characters.
Context window
The maximum number of tokens the model can consider at once: your conversation, attached documents and hidden instructions combined. A 200K-token window holds a fat novel. Everything outside the window is invisible, which is exactly why a model “forgets” the start of very long chats. It didn’t forget. It never saw it.
Prompt
Everything you send to the model. There’s a hidden instruction layer set by the app developer (the system prompt) and your message (the user prompt). Here’s the thing nobody expects: better prompts, with context, examples and constraints, beat better models surprisingly often.
Temperature
A dial controlling randomness. At zero, the model picks the most likely continuation every time: deterministic, safe, sometimes dull. Crank it up and the model takes more risks: more creative, more chaotic. Doing factual work? Keep it low. Brainstorming? Turn it up and enjoy the chaos.
Hallucination
A fluent, confident, false statement. Models generate plausible text rather than verified truth, so gaps in knowledge get filled with invention. The defense is simple, if unglamorous: verification, and grounded tools that cite sources.
How models are made
Pretraining
The expensive first phase, where the model learns language and knowledge by predicting text across a giant corpus. This is where capabilities come from, and where the nine-figure compute bills live.
Fine-tuning
Additional training on curated examples to change behavior: your tone, your format, your classification scheme. Cheap compared to pretraining, powerful when you have good examples, and (say it with us) the wrong tool for teaching facts.
RLHF and preference tuning
Post-training where human or AI reviewers rank answers, teaching the model to be helpful, harmless and pleasant. This phase is why assistants have personalities. And refusals.
Distillation
Training a small, fast model to imitate a big one. This is how you end up with a model that runs on a laptop while keeping a large fraction of a frontier model’s everyday skill.
Open weights
A model you can download and run yourself, like the Llama, Mistral or DeepSeek releases. Not quite “open source” in the classic sense, but close enough for most practical purposes.
How models are used
Inference
Using the model, as opposed to training it. Every chat message you send is an inference call. Inference cost and latency, not training, dominate the economics of AI products.
RAG (retrieval augmented generation)
Fetching relevant documents at question time and stuffing them into the prompt, so the model answers from your data instead of its memory. This is the standard architecture for chatbots over company knowledge, and we break it down fully in our RAG, fine-tuning and agents guide.
Embeddings
Numeric fingerprints of text meaning, used to find similar documents. Embeddings power the retrieval in RAG and semantic search. It’s why “refund” matches “money back”: their embeddings sit close together.
Agent
A model in a loop with tools. It plans, calls search or APIs or code, reads the results, and keeps going until the task is done. Powerful, brittle, and the industry’s current obsession.
Multimodal
A model that handles more than text: images, audio, video, sometimes generation too. Asking questions about a photo or dictating to your assistant is multimodal AI in action. Curious? See our deep dive on multimodal models.
How models are judged
Benchmark
A standardized test for models: MMLU for knowledge, HumanEval for code, and so on. Useful, gameable, and routinely overinterpreted in marketing. Our benchmark reading guide shows you how not to be fooled.
Leaderboard
A ranked comparison, most famously LMArena, where humans vote on blind answers. Closer to real preference than static benchmarks, though it has its own biases toward confident, nicely formatted answers.
Latency and throughput
How fast the first token arrives, and how many tokens per second follow. For chat, latency shapes how good a model feels more than most benchmark points do.
Reasoning model
A model trained to think step by step internally before answering, spending extra compute for better answers on hard problems. The biggest capability jump of the last two years.
System card
The document a lab publishes alongside a model: capabilities, evaluations, safety results. Read these instead of the launch tweets. The caveats live here.
Our editorial promise. Every definition here reflects how practitioners actually use these terms in 2026, checked against primary documentation. We update this glossary as usage shifts. More on our methodology page.
The next terms you’ll meet
Once these twenty feel comfortable, here’s what’s arriving in your feeds next. Function calling and its descendant MCP, the model context protocol: standardized ways for a model to use external tools and data sources, the plumbing behind every agent. Quantization: compressing a model’s numbers so it runs on smaller hardware, the trick that puts capable models on laptops. Mixture of experts: an architecture where only part of the model activates per token, delivering big-model quality at small-model speed; most frontier models now work this way. Guardrails: the filters and rules that block harmful or off-policy outputs. Context caching: reusing processed prompt prefixes to cut cost and latency on repeated long inputs. Evals: shorthand for evaluations, the test suites that decide what ships.
You don’t need to master these to use AI well. You need to recognize them, so the next product announcement reads as engineering choices rather than magic words.
The bottom line
Remember that pitch meeting from the beginning? Next time a vendor claims a million-token context window, you’ll ask how retrieval quality holds up in the middle. Next time someone pitches a fine-tune, you’ll ask whether RAG solves it cheaper. That’s what vocabulary really is: leverage. It converts mystique into questions with answers.
Keep the momentum going in our LLM section, where every one of these terms shows up in real, practical context.