Everyone treats large language models like a black box, which makes them harder to work with than they need to be. Here's the mental model, without the math, that actually makes prompting, context management, and hallucination predictable instead of mysterious.
It starts with tokens. Text gets broken into small chunks — not always whole words — and each chunk is mapped to a point in a high-dimensional space called an embedding. Words and phrases with similar meaning end up near each other in that space; that's the whole trick that lets the model generalize instead of memorizing.
The mechanism that makes context work is attention. At every layer, every token looks at every other token in the input and asks, in effect, "how relevant are you to me right now?" Stack that operation dozens of layers deep and the model builds up a surprisingly rich representation of what the text actually means, not just what words it contains.
Pretraining is deceptively simple: the model is trained on one objective — predict the next token — over an enormous amount of text. That single objective, repeated at scale, is enough for the model to pick up grammar, facts, reasoning patterns, and code, the same way a person absorbs a language by being immersed in it rather than studying its grammar rules.
Raw pretraining gets you a very good autocomplete engine, not an assistant. Fine-tuning and RLHF (reinforcement learning from human feedback) reshape that raw model to follow instructions and prefer answers people actually rate as helpful and honest — the step that turns "predicts the next word" into "answers your question."
At inference time — when you're actually chatting with it — the model reads your prompt as tokens and predicts the next token, over and over, sampling from its own probability distribution each time. Everything it "knows" for that conversation is whatever fits inside the context window; nothing more, nothing less.
Once you see an LLM as next-token prediction on top of patterns learned from enormous amounts of text, most of what looks like magic — or unpredictability — turns into something you can reason about and design around, which is the whole point of building agents on top of one instead of just chatting with it.