LLM Mini Lab

Station 1

How does an LLM learn?

Use one student character to separate training from generation: broad reading, teacher guided examples, and practice with feedback.

Three broad stages in training modern LLMs Actual pipelines differ, but this mental model is a strong beginner map.

Pretraining builds broad language patterns. Fine tuning teaches answer style. Feedback helps improve behaviour.

Student, teacher, token, and feedback learning collage
Read enormous amounts of material

Pretraining

The student reads a huge library and gradually discovers patterns in language.

Large amounts of text -> Predict missing or next tokens -> Learn language patterns
Student learning Read -> Learn how to answer -> Improve through feedback
LLM training Pretraining -> Supervised Fine Tuning -> Reinforcement Learning

Station 2

Words become tokens

Begin with ordinary text. A language model does not receive the sentence as a single human-like thought; it receives a sequence of token pieces.

Token A small unit of text that a language model processes.

LLMs do not directly see sentences the way humans do. They process sequences of tokens.

Artificial intelligence is changing education.

Press Tokenize to break the sentence into model-sized pieces.

Station 3

Tokens enter the context

The context is the text currently available to the model while it predicts the next token. It is working information, not human memory.

Context

The text currently available to the model when generating its next token.

What is in the context strongly shapes what the model predicts next.

Context Window 0 tokens
Prompting connection Adding clearer context changes the prediction space the model sees.

Station 4

A large neural network processes the context

The neural network idea from the previous lab now operates at enormous scale. Tokens flow through learned parameters and become probabilities.

Parameters

Values learned during training that encode patterns discovered from data.

The earlier learning station explains how pretraining, supervised fine tuning, and feedback shape these learned parameters.

Learned probabilities Waiting
Training Data Examples from text
Parameters Learned patterns
Inference Use the learned model

Station 5

Predict the next token

This is the central LLM move: given the current context, the model estimates which token is most likely to come next.

Core Insight An LLM generates text by predicting which token should come next given the current context.
Current context The capital of India is

Station 6

One token becomes many

A response appears because the selected token is added back into the context. Then the model predicts again. The loop repeats.

Context Predict token Add token Repeat

Neural network learning: Predict -> Compare -> Adjust -> Repeat. LLM generation: Context -> Predict token -> Add token -> Repeat.

Updated context
Generated response

AI

Station 7

The surprise: fluent text can still be wrong

If the model is predicting likely text, does that guarantee the answer is true?

Hallucination

A plausible sounding but incorrect or unsupported response generated by an AI model.

Generating plausible language is not the same as verifying truth.

User asks Who invented the solar-powered pencil in 1882?

Station 8

The whole LLM mental model

Pretraining Fine tuning Feedback Learned parameters
User context Neural network Next token probabilities Selected token Updated context Generated response

Knowledge in parameters

Patterns learned during training. These may be useful, outdated, incomplete, or wrong.

Knowledge in context

Information currently provided to the model, such as your prompt, examples, or uploaded material.

Response

Learned patterns plus current context, expressed through repeated next-token prediction.

Training: Pretraining -> Fine tuning -> Feedback. Generation: Text -> Tokens -> Context -> Next token probabilities -> Response -> Hallucination risk.