How Language Models Generate Text

A language model can turn a short request into a convincing travel plan. That fluency is useful for organizing information, but an authoritative tone does not establish whether a train route exists or a restaurant is still open.

This tutorial separates the model from the application around it. We use a fictional Helsinki itinerary to trace that boundary before applying it to the OptiVerse Travel copilot in later lessons. The itinerary is a teaching fixture, not a recorded experiment or current travel advice.

Worked Example: A Fluent Itinerary with Unsupported Details

Suppose a user asks:

Plan a trip to Helsinki.

Our deliberately flawed sample answer includes three claims:

Take a regular VR passenger train directly to Porvoo. In late June, watch the sun remain above the horizon all night in Helsinki. In October, count on seeing the northern lights during your visit.

Checking the relevant sources overturns all three:

• Porvoo transport: Visit Porvoo describes frequent buses from Helsinki alongside a seasonal museum railcar service. A summer museum railcar does not establish regular VR passenger service; you still need the operator's dated timetable before choosing a departure. Visit Porvoo

• Summer daylight: Helsinki has bright white nights in summer, but the sun still sets. White nights and the midnight sun north of the Arctic Circle, where the sun stays above the horizon all night, are different claims. Visit Finland

• Auroras: Finland's southern coast sees auroras far less often than Lapland. In its guidance for dark, cloudless nights, the Finnish Meteorological Institute cites a rough long-run rate of about once a month on the south coast. That is a regional statistic, not a trip forecast or guarantee. FMI

These travel sources were checked on 10 September 2026 and rechecked on 2 November 2026; actual timetables and viewing conditions still need checking for the specific travel date.

The fixture shows that three claims fail against external sources. It does not show how a particular model produced them: the final text alone cannot identify the training example or internal cause behind the wording.

From Training to Generation

Training and parameters

A neural network is a numerical function whose adjustable values (weights and, in many architectures, bias terms) are called parameters. Training updates those parameters to optimize an objective over data.

In autoregressive language models, a central pretraining objective is predicting the next token from preceding tokens. Optimizing that objective can support factual recall, linguistic structure, and problem-solving patterns, though naming the objective does not by itself explain every capability it produces. GPT-3 is a documented example of this autoregressive family, which is the scope of this lesson rather than every possible language-generating architecture. Brown et al., 2020

Parameters can encode factual associations, yet weights are not a database of records stamped with dates and citations. A fact recalled from parameters carries no built-in proof that it remains current, and a training cutoff describes a limit on the training data rather than guaranteeing complete or accurate knowledge of everything before that date.

Tokens and inference

A tokenizer maps text into a sequence of integer token IDs. Depending on the tokenizer, a single token might cover a whole word, a subword, punctuation, whitespace, or byte fragments. English word counts do not predict Traditional Chinese token counts reliably, so measure with the selected model's tokenizer instead of guessing how a word like "Helsinki" will split.

Inference runs a trained model on new input; in an ordinary inference request, its parameters stay fixed. Even without weight updates, the model can adapt its output to examples or facts supplied in the prompt. This is in-context learning, which GPT-3 demonstrated without gradient updates on the evaluated tasks. Brown et al., 2020

Training changes the numerical function itself; context changes the input passed into that function. Surrounding product code may store conversation history or collect data for future training runs, whereas a single inference call performs neither on its own.

A small probability example

In a simplified autoregressive decoder, the model reads the current context and outputs scores across its vocabulary for the next token. A decoding rule selects one token; that token is appended to the context, and the loop repeats until a stopping condition is reached.

For illustration, suppose the next-token distribution at one step is:

The probabilities sum to 1.00. Greedy decoding picks bus, whereas sampling can pick train or ferry. These numbers are illustrative, not measured from a model. Even so, a 0.50 probability for bus describes how strongly the decoding distribution favors that token next; it is not a 50% chance that the resulting travel advice is true.

Conceptually, generation runs text → token IDs → model scores → decoding choice → output tokens → text. Production serving engines optimize how those steps execute, while the gap between choosing plausible tokens and verifying external facts remains unchanged.

Five Layers of an Assistant

The five-layer map below is an editorial teaching aid. Real systems can combine layers, and one user-visible behavior can have several contributing causes.

Post-training matters: in the InstructGPT study, human raters preferred a much smaller post-trained model over base GPT-3 on the authors' prompt distribution. That is evidence about that evaluation, not a guarantee that every post-trained model is more capable on every task. Ouyang et al., 2022

When a tool fetches a live timetable and the application inserts it into the prompt, two layers cooperate: retrieval/tooling obtains current external evidence, and request context exposes it to the model without changing a single weight.

Why Fluent Answers Can Be Wrong

Engineers often use hallucination as shorthand for false or unsupported output, yet fixing a system requires naming the concrete failure: an answer may contradict reality, misrepresent supplied evidence, combine valid facts into an invalid conclusion, or state an unverified guess with certainty.

Next-token prediction is not a fact-checking mechanism, nor is the pretraining objective the sole cause of errors. Training data, post-training choices, underspecified prompts, missing context, reasoning slips, and decoding choices can all contribute; our Helsinki fixture illustrates the symptom without isolating an internal cause.

Using a larger model is no substitute for verification either. On the TruthfulQA benchmark, larger models among those tested were often less truthful on questions designed to trigger common human misconceptions. Treat that as a benchmark-specific warning against equating scale with factual reliability: it does not show that scaling always makes models worse, and it does not describe the performance of today's models. Lin, Hilton, and Evans, 2022

The itinerary mixes two kinds of verification need. Whether the sun sets in Helsinki in late June is relatively stable geography; whether a specific bus or train runs today depends on an operational schedule. Both claims need evidence, but only the schedule requires a date-specific live lookup. Calling every error "outdated knowledge" blurs that engineering distinction.

Build Artifact and Hands-On Exercise

Create a worksheet with columns item, primary layer, reason, and external evidence needed. Classify these ten items by primary owner using the five-layer map:

1. Learning text patterns by updating parameters during pretraining.

2. Producing next-token scores during inference.

3. Learning instruction-following behavior from supervised demonstrations after pretraining.

4. Refining behavior using preference feedback after pretraining.

5. Including "I am vegetarian" in the current request.

6. Including a formatting example in the current request.

7. Fetching a dated transport timetable from its operator.

8. Sending an authorized booking request to the booking service.

9. Rejecting a booking because the principal lacks permission.

10. Enforcing a rule that current departure claims require dated transport evidence.

Worked row: item 7 belongs to retrieval/tooling because the system queries an external source; before that result can support an answer, it still has to match the requested route, travel date, and operator identity.

Deliberately Broken Case

Suppose the application treats a summer museum railcar page as proof of regular passenger service today. The generated reply quotes the source wording accurately while applying it to the wrong service type and date.

Record source wording and claim support separately. Repairing the failure means either verifying the exact service and travel date or stating clearly that today's departure has not been confirmed; merely hedging the tone leaves the evidence gap intact.

Exit Criteria

Classify at least 8 of the 10 worksheet items correctly, state the difference between next-token probability and factual reliability, and explain why the museum-railcar page fails to support today's regular passenger departure. Check and revise any missed rows against the answer key.

Knowledge Check

1. Does adapting an answer to a new fact in the prompt require updating model parameters?

2. Can an incorrect itinerary by itself prove which training document caused the mistake?

3. Which claims in the Helsinki example require checks tied to the specific travel date?

Answer Key

Worksheet owners: 1–2 base model; 3–4 post-training; 5–6 request context; 7–8 retrieval/tooling; 9–10 application policy. Multiple layers can interact at runtime, while each row names a single primary responsibility.

Knowledge check: (1) no, because new prompt context changes the input to fixed parameters; (2) no, because surface output alone cannot establish training provenance or internal cause; (3) transport departures and aurora conditions require date-specific checks, whereas Helsinki's summer sunset requires accurate geographic reference material rather than a live transit query.

Where This Goes Next

Next, build a request and context-budget worksheet to see how instructions, examples, and retrieved evidence fit inside a finite context window, and how to tell missing context apart from ignored context.

Sources and Further Reading

• Brown et al. (2020), Language Models are Few-Shot Learners. Autoregressive GPT-3 and in-context evaluation without gradient updates.

• Ouyang et al. (2022), Training Language Models to Follow Instructions with Human Feedback. Post-training and the scope of human-preference gains on the evaluated prompt set.

• Lin, Hilton, and Evans (ACL 2022; preprint 2021), TruthfulQA. Benchmark evidence that model scale alone does not guarantee truthfulness.

• Visit Porvoo, Arriving. Transport categories between Helsinki and Porvoo; verify individual departures separately.

• Visit Finland, Land of the Midnight Sun. White nights in southern Finland versus midnight sun in the north.

• Finnish Meteorological Institute, Auroras in Finland. Long-run regional viewing statistics rather than a trip-specific forecast.