How Large Language Models Actually Work: A Plain-English Guide

Large language models can draft contracts, explain code, and hold a conversation that feels surprisingly human. Underneath, the core mechanism is simple enough to describe in a sentence, and the complexity comes from scale and from several layers of training stacked on top. This guide explains those layers in plain terms so you can reason about what these systems do well, where they break, and why.

The one-sentence version

A large language model (LLM) is a program that, given some text, predicts what text is likely to come next. It does this one small piece at a time, adds that piece to the text, and repeats.

Everything else, including answering questions, following instructions, writing poems, and refusing harmful requests, is built on top of that loop. Keeping the loop in mind explains a lot of behavior that otherwise seems mysterious.

Step one: turning text into numbers

Computers do not process words directly. Before a model sees anything, text is split into tokens, which are chunks that might be whole words, parts of words, punctuation, or spaces. A common word may be a single token, while a rare word or a long product name may be split into several.

Each token maps to an ID in a fixed vocabulary, and each ID maps to a list of numbers (a vector) that the model learns during training. These vectors are the model's internal representation of the token. At the start of training they are random. By the end, tokens used in similar ways tend to have vectors that relate to each other in useful patterns.

This is why models sometimes stumble on tasks like counting letters in a word: they never see individual letters, only tokens that bundle letters together.

Step two: pretraining on next-token prediction

The first and most expensive phase is pretraining. The model is shown an enormous amount of text: books, articles, websites, code, and other sources. The training task is always the same. Hide the next token, ask the model to predict it, measure how wrong the prediction was, and nudge the model's internal numbers (its parameters, often called weights) so that next time it is a little less wrong.

A worked example makes this concrete. Imagine the training text contains:

The capital of France is Paris.

The model sees "The capital of France is" and produces a probability for every token in its vocabulary. Early on, it might spread probability almost evenly. The training process compares that output with the real next token, "Paris," and adjusts billions of parameters slightly so that "Paris" gets more probability in similar contexts. Repeat this across a vast number of examples and the model gradually absorbs the patterns of language.

Why such a simple task produces so much capability

Predicting the next token well is harder than it sounds. To guess the next word in a legal brief, a model benefits from representing legal concepts. To continue a Python function, it benefits from tracking which variables exist and what they hold. To finish a sentence in a mystery novel, it benefits from tracking characters and motives.

Nobody programs these abilities in. They emerge because they reduce prediction error. The model is under constant pressure to build internal structure that captures grammar, facts, style, and some forms of logic, because all of those help it guess what comes next.

The architecture, briefly

Modern LLMs are almost all based on the transformer architecture. Its key idea is attention: when processing a token, the model can look back at every earlier token in the input and decide which ones matter most. In "The trophy did not fit in the suitcase because it was too big," attention lets the model connect "it" to "trophy" rather than "suitcase."

Transformers stack many layers of attention and simple computation. Earlier layers tend to capture local patterns, and later layers tend to build more abstract relationships. You do not need the math to use these models, but it helps to know that the model's "understanding" of any token is shaped by the other tokens around it.

What a pretrained model is (and is not)

After pretraining, you have what is often called a base model. It is extremely good at continuing text, but it is not an assistant.

If you give a base model the text "How do I reset my router?", it might answer. It might also continue with "How do I change my Wi-Fi password? How do I update firmware?" because on the internet, questions often appear in lists. It has learned what text looks like, not what a helpful response is.

A base model also has no built-in sense of which continuations are appropriate. It has seen helpful text and harmful text, accurate text and misinformation, and it will reproduce whatever pattern the prompt most resembles.

Step three: instruction tuning

To turn a base model into something that follows directions, developers run instruction tuning (also called supervised fine-tuning). They collect many examples of an instruction paired with a high-quality response, such as:

• A request to summarize a paragraph, followed by a good summary
• A coding question, followed by a correct and well-explained answer
• An ambiguous request, followed by a clarifying question

The model is trained on these pairs using the same next-token machinery as before. The difference is the data. Now the patterns it learns are "when text is formatted as a user request, the next text is a direct, helpful response."

This phase is much smaller than pretraining, but its effect on behavior is large. Most of the knowledge already exists in the base model. Instruction tuning mainly teaches format, tone, and the habit of actually addressing the request.

Step four: learning from preferences

Instruction tuning gets a model to respond helpfully, but writing ideal responses for every situation is slow, and many qualities are easier to judge than to write. That is where preference-based training comes in. The best-known version is reinforcement learning from human feedback (RLHF).

At a conceptual level, it works like this:

1. The model generates several responses to the same prompt.
2. People compare those responses and indicate which is better, based on guidelines about helpfulness, accuracy, and safety.
3. Those comparisons train a separate reward model that learns to score responses the way the human raters would.
4. The language model is then trained to produce responses that the reward model scores highly, while being kept from drifting too far from its earlier behavior.

Variants of this idea exist. Some skip the separate reward model and learn directly from preference pairs. Some use AI-generated feedback guided by written principles in place of, or alongside, human ratings. The shared goal is to shape behavior using judgments of "this answer is better than that one."

What preference training changes

Preference training is where a model picks up many of the traits people associate with assistants: admitting uncertainty, declining clearly harmful requests, structuring answers readably, and staying on topic. It also introduces its own side effects. If raters tend to prefer longer or more confident answers, the model can learn to be verbose or to sound sure when it should not. Tuning this balance is an ongoing challenge.

How generation actually happens

When you send a prompt, the model runs through a loop:

1. Convert the prompt to tokens.
2. Compute a probability for every possible next token.
3. Pick one token from that distribution.
4. Append it to the text and go back to step 2.
5. Stop when a special end token is produced or a length limit is hit.

Step 3 is where settings like temperature matter. At low temperature, the model almost always picks the most likely token, producing consistent and conservative output. At higher temperature, less likely tokens get picked more often, producing more varied and sometimes more creative output, with a higher chance of going off track.

Two consequences follow from this loop:

• The model commits as it goes. Once it writes the first sentence of an answer, every later token is conditioned on that sentence. An early mistake tends to get built upon rather than corrected.
• The same prompt can give different answers. Because tokens are sampled, two runs can diverge at any point and end up in different places.

Why models seem to reason

Ask a capable model a multi-step word problem and it often gets it right, explaining each step. Is that reasoning?

The honest answer is that it depends on your definition, but the mechanism is informative. During pretraining, the model saw huge amounts of text in which people worked through problems step by step: textbooks, forum answers, code comments, proofs. Producing text that follows those patterns requires internal computations that track intermediate results. Those computations are functionally similar to reasoning, even if they are not structured the way human reasoning is.

A few practical observations support this view:

• Writing out steps helps. Models often do better when they work through a problem before giving a final answer. Each written step becomes part of the context, so the model can build on it rather than trying to compute everything in a single token.
• Familiar structure helps. Problems that resemble common patterns are solved more reliably than problems with unusual structure, even if the underlying logic is identical.
• Small changes can break it. Rewording a problem, adding irrelevant details, or changing names can sometimes lower accuracy, which suggests the model relies partly on surface patterns.

Some newer models are trained specifically to generate longer internal chains of reasoning before answering, and to check their own work. This tends to improve results on math, coding, and planning tasks, at the cost of more tokens and more time per answer.

The limits that follow from the design

Understanding the training process makes the common failure modes predictable rather than surprising.

Fluency is not accuracy

The model is optimized to produce plausible text. Most of the time, plausible and true overlap. When they do not, such as for an obscure fact, a specific citation, or a number the model never reliably learned, it can produce a confident, well-formatted answer that is simply wrong. The mechanism that makes output smooth is the same mechanism that makes errors hard to spot.

Knowledge is frozen and uneven

A model knows only what was in its training data, up to a cutoff point. It has no automatic awareness of events after that. Its knowledge is also uneven. Topics that appear frequently in text are well represented. Niche topics, internal company information, and recent developments are weak or missing unless you supply them in the prompt.

Limited working memory

A model can only attend to a fixed amount of text at once, called its context window. Anything outside that window is invisible. Even inside the window, very long inputs can make it harder for the model to use details reliably.

No built-in grounding or verification

A plain LLM does not look anything up or run checks unless it is connected to tools that do. It cannot tell the difference between something it "knows" and something it is pattern-matching into existence. Systems that add search, databases, calculators, or code execution exist precisely to cover this gap.

Sensitivity to phrasing

Because output depends on the full input text, small changes in wording, order, or formatting can shift results. This is why clear prompts and consistent templates matter in production systems.

A mental model you can use

When working with an LLM, it helps to picture it as a very widely read collaborator with an excellent sense of how text usually goes, no access to the outside world unless you give it one, and a strong tendency to finish whatever pattern you start.

That picture leads to practical habits:

• Give it the information it needs instead of assuming it knows.
• Ask it to show its work on multi-step problems.
• Check specific facts, numbers, and references independently.
• Treat a single output as a sample, not a guarantee.
• Design your system so that errors are caught before they reach users.

Key takeaways

• An LLM predicts the next token repeatedly; nearly every capability is built on that loop.
• Pretraining on large text collections gives the model broad knowledge and language skill, while instruction tuning and preference training turn it into a usable assistant.
• Reasoning-like behavior emerges because tracking intermediate steps helps predict text, and writing steps out usually improves results.
• Sampling makes output variable, and early mistakes tend to compound as the model continues.
• Core limits, including confident errors, frozen knowledge, and finite context, follow directly from how models are trained and should shape how you build with them.