How AI Models Learn: What This Guide Covers

Part 1 of this series covered what a model actually is: a learned recipe, tuned by parameters, doing inference once training is done. Fair question to ask next: learned from what, exactly, and how do you stop it learning the wrong lesson?

This post is about the learning itself — training data, epochs, loss, overfitting, fine-tuning, and RLHF. Six terms that show up constantly in AI writing and are rarely defined. I will not assume you know any of them.

What Is Training Data in Machine Learning?

Training data is the set of examples a model studies during training. For a language model, that is an enormous amount of text — books, websites, code, conversations. For an image classifier, it is photos with labels attached (“this one is a cat”). Google’s Machine Learning Glossary defines the closely related term “training set” the same way: the subset of the dataset a model actually learns from, as opposed to the data held back to check it afterwards.

The quality of that pile matters more than people expect. Feed a model sloppy, biased, or narrow examples, and it learns sloppy, biased, narrow habits — the same way you would if the only cookbook you ever studied was badly translated. Rubbish in, rubbish out is not a joke here; it is the entire mechanism.

What Are Epochs and Loss in Model Training?

An epoch is one full pass through the training data — the model has looked at every example once. Training usually runs for many epochs, not one, because a single read-through is rarely enough to learn much (I certainly cannot learn a new language from reading a textbook exactly once). Google’s Machine Learning Glossary defines an epoch the same way: a full training pass over the entire training set, such that every example has been processed once.

Loss is a numeric measurement of how wrong the model’s current predictions are; training exists to push that number down. Loss starts high, because the model is guessing blindly. Then, epoch by epoch, the parameters get nudged to make the same mistakes a little less often, and loss creeps down.

loss
 |●
 |  ╲
 |    ╲___
 |         ‾‾‾●___
 +-------------------→ epochs
 high loss        low loss

That falling curve is genuinely most of what “training” means in practice: guess, measure how wrong, adjust, repeat, thousands or millions of times.

What Is Overfitting in Machine Learning?

Overfitting is a model behaviour where the model has memorised its training examples instead of learning the general pattern behind them. Imagine a student who prepares for an exam by memorising last year’s paper word for word, rather than understanding the subject. They will ace that exact paper and fail anything slightly different — that is overfitting.

An overfit model looks brilliant on the data it was trained on and falls apart on anything new — which is precisely why it is trained on one set of data and checked against a separate set it has never seen. Google’s Machine Learning Crash Course describes an overfit model as an invention that performs well in the lab but is worthless in the real world — an apt summary of the training-loss-down, validation-loss-up pattern in the table above.

What Is Fine-Tuning in AI Models?

Fine-tuning is the process of further training an already-trained, general-purpose model on a smaller, more focused set of examples so it gets noticeably better at one particular thing. Training a large model from nothing is enormously expensive, so most of the time nobody does it twice — fine-tuning is the shortcut. Hugging Face’s documentation describes the same trade-off: fine-tuning a pretrained model reduces compute cost and lets you use a state-of-the-art model without training one from scratch.

Think of a chef who trained across every cuisine there is, then spent a further month working exclusively in an Italian kitchen. Still the same chef, same broad skill set underneath, but now unmistakably sharper at pasta.

What Is RLHF (Reinforcement Learning from Human Feedback)?

RLHF stands for Reinforcement Learning from Human Feedback, and it is the step that turns a raw, fine-tuned model into something you would actually want to talk to. Humans rate a model’s answers — this one is helpful, that one is rude or wrong — and those ratings train the model to produce more of the answers people rate highly.

RLHF resembles a diner sending a dish back — not because the recipe was technically wrong, but because it was not what anyone wanted to eat. RLHF is a large part of why a raw model can sound blunt or erratic while the assistant version built on top of it sounds considerably more polite and useful.

A Combined Analogy: Training Data, Epochs, Loss, Overfitting, Fine-Tuning, and RLHF

Training data is what the chef studies. Epochs are how many times they go through the material. Loss is how far each attempt is from correct. Overfitting is memorising a recipe instead of learning to cook. Fine-tuning is specialising. RLHF is a long string of diners sending dishes back until the chef learns what people actually want.

How AI Models Learn: Quick-Reference Checklist

1. Training data — the examples a model learns from; quality shapes everything downstream.
2. Epoch — one full pass through the training data.
3. Loss — a number measuring how wrong the model currently is; training pushes it down.
4. Overfitting — memorising training examples instead of learning the general pattern.
5. Fine-tuning — further training a general model on a narrower set of examples to specialise it.
6. RLHF — using human ratings of a model’s answers to train it toward more helpful behaviour.

Next in This Series: Key Machine Learning Concepts

Training data, epochs, loss, overfitting, fine-tuning, and RLHF do not need maths to understand — training is just guess, measure, adjust, on repeat, at a scale that is hard to picture. What matters is knowing overfitting from genuine learning, and fine-tuning from RLHF, because release notes use all four words assuming you already do.

Next in the series, I bring in the actual maths: Part 3 covers the algorithms underneath — regression, gradient descent, and the classic architectures — for anyone who wants to go one level deeper.

References

• AI Terms 101: The Building Blocks — Part 1 of this series
• Training language models to follow instructions with human feedback — OpenAI’s InstructGPT paper, the paper that popularised RLHF for language models
• Overfitting — Google’s Machine Learning Crash Course definition and loss-curve diagnosis
• Machine Learning Glossary: ML Fundamentals — Google’s definitions of training set, epoch, and loss
• Fine-tune a pretrained model — Hugging Face’s documentation on fine-tuning