How does a large language model work, end to end?

Two loops explain the whole pipeline

An LLM is easiest to understand as two related loops.

The training loop processes large collections of token sequences, measures how badly the model predicts their continuations, and adjusts billions of numerical parameters called weights. Training produces a model artifact: learned weights plus a compatible tokenizer and configuration.

The generation loop keeps those weights fixed. It turns the current context into scores for the next possible token, selects one token, appends it to the context, and runs again. A paragraph, code patch, or tool request emerges one token at a time.

flowchart TB
  subgraph Learn[Offline: learn the model]
    D[Curated text and code] --> TK[Token sequences]
    TK --> P[Predict continuations]
    P --> L[Measure loss]
    L --> B[Backpropagate gradients]
    B --> W[Update weights]
    W --> P
  end
  subgraph Use[Online: generate with fixed weights]
    C[Current context] --> I[Token IDs and embeddings]
    I --> T[Transformer blocks]
    T --> S[Next-token scores]
    S --> X[Decoder selects a token]
    X --> C
  end
  W --> T

These loops share the same neural network but do different work. Training changes the network. Inference uses it. A conversation usually changes the next input, not the learned weights.

One generated token, from text to text

Follow one pass through the model:

1. Encode the input. Tokens and tokenization explains how text and control markers become vocabulary IDs. Token boundaries depend on a specific tokenizer; they are not necessarily words.
2. Create initial representations. Embeddings turn each ID into a learned vector. Positional information tells the model where that token appears.
3. Mix information through the network. Transformers and attention repeatedly update every position. Attention moves selected information between positions; feed-forward networks transform each position; residual paths preserve and accumulate updates.
4. Produce vocabulary scores. The final representation at the prediction position is projected into one score, or logit, for every possible next token.
5. Choose one continuation. Generation and sampling turns logits into probabilities and applies a decoding policy such as greedy selection or temperature-based sampling.
6. Append and repeat. The chosen token joins the context. The model performs another forward pass until an end token, length limit, protocol boundary, or application stop rule ends generation. Serving systems normally reuse a KV cache so previous attention state is not recomputed from scratch at every step.

The model does not normally write a complete answer somewhere and reveal it gradually. Each selected token changes the context for every token that follows.

Where the behaviour in the weights comes from

During pretraining, the target already exists in the data: for each position, the next observed token is the label. The model performs a forward pass, cross-entropy loss penalizes insufficient probability on the observed continuation, backpropagation computes gradients, and an optimizer updates the weights. Repeating this over many batches teaches reusable statistical structure in language, code, and other sequences.

Model training explains that optimization loop. Fine-tuning and preference training covers later stages that make a base model more useful for instructions, dialogue, tools, or a narrower task. Those stages may use demonstrations, preference comparisons, reward models, direct preference objectives, reinforcement learning, or adapters.

Next-token prediction can support surprisingly broad behaviour because doing it well requires representing syntax, references, patterns, facts, formats, and task structure. It still optimizes prediction under training and post-training signals—not truth, authorization, or successful completion in a real environment.

The same model can produce different answers

Keep four sources of variation separate:

• Weights and tokenizer: what was learned and how text maps to IDs.
• Context: system and user instructions, conversation, retrieved passages, tool definitions, and tool results supplied for this request.
• Decoder: temperature, top-p, constraints, seed, maximum length, and stopping rules.
• Surrounding application: retrieval, memory, tools, policy, retries, safety checks, and presentation.

This map makes debugging much faster. Wrong token boundaries point to the tokenizer. Missing or stale evidence points to context construction or retrieval. Repetition and truncation may point to decoding or stop rules. An unauthorized action is a harness and application failure even if the model proposed it fluently.

Where an LLM ends and an AI system begins

An LLM maps a supplied context to a distribution over continuations. It does not independently search the current web, remember durable business state, authenticate a user, execute a shell command, approve a payment, or verify that a code change works.

An application can add those capabilities around the model. A retrieval system can place current evidence in context. An AI agent can repeat model calls around tools and observations. An agent harness owns that loop's state, permissions, execution, failure handling, and stopping rules.

This boundary explains an apparent paradox: two products can use the same model and behave very differently. The model is important, but the context and runtime determine what evidence it sees, what actions are available, and which proposals can become real effects.