Introduction to LLMs: tokens & context

Lesson 1 of 8 · 20 min · part of the AI Prompt Engineer certification

A large language model (LLM) is a neural network trained on enormous amounts of text to do one thing extremely well: predict the next token. Everything it does — answering questions, writing code, summarizing — emerges from repeating that prediction, one token at a time.

To use LLMs well you need an accurate mental model of three things: what a token is, what the context window is, and how the model picks the next token. This lesson covers the first two.

Tokens: the unit models read

Models do not see characters or words. Text is split into tokens — common words, word fragments, punctuation and whitespace. In English, one token is roughly 4 characters or ¾ of a word. Rare words, numbers, code and non-English text usually take more tokens.

Tokens matter because they are how you are billed, how latency scales, and how the context window is measured.

The context window

The context window is the maximum number of tokens the model can consider at once — your system prompt, the conversation history, any documents you paste in, and the tokens it generates in response.

LLMs are stateless: they remember nothing between API calls. A chat interface "remembers" your conversation only because the whole history is resent on every turn. When the history grows beyond the window, something has to be dropped or summarized.