Teaching the
machine.
Training is where billions of blank dials become a model that can write. It's the expensive, one-time-ish part — and under the hood it's a surprisingly simple loop repeated an astronomical number of times.
A fresh model knows nothing — its billions of weights are effectively random. Training is the process of nudging every one of those weights until the model's guesses match reality. Here's the loop.
Guess, check, adjust — a trillion times
Training walks through mountains of text and, at each step, does four things:
- Predict. Given some text, the model guesses the next token.
- Compare. It checks its guess against the token that actually came next.
- Score the error. The gap between guess and reality becomes a single number called the loss. Bad guess → high loss. Good guess → low loss.
- Adjust. It nudges every weight a tiny amount in the direction that would have made the loss lower (this is backpropagation and gradient descent — fancy names for "figure out which dials to turn, and turn them a hair").
Repeat that billions of times across the whole dataset and the loss slowly falls. A falling loss is the whole game — it's the model getting measurably better at predicting real human text.
In the terminal below, the loss column drops from 4.8 to 1.6 across three passes over the data. That number going down is learning — you're literally watching the dials find better settings.
Why it takes so much
Each full pass over the training data is an epoch. Real frontier models train on datasets measured in trillions of tokens — a large fraction of the public internet, books, and code — and every single token triggers that predict‑compare‑adjust step across every weight. That's why training the biggest models takes thousands of specialized chips running for weeks and costs millions of dollars.
Data quality matters as much as quantity. A model only ever learns the patterns in what it's shown — feed it junk and it learns junk. "Garbage in, garbage out" is not a slogan here; it's mechanical.
Two very different price tags
Building the brain
Training from scratch on a massive general dataset. This is the multi-million-dollar, weeks-on-thousands-of-GPUs part. Only a handful of labs do it.
Specializing it
Taking an existing trained model and nudging it on a small, focused dataset to teach a style, domain, or task. Cheap and fast by comparison — hours, not weeks.
For almost every business, pretraining is irrelevant — you start from a capable open or frontier model and, at most, fine-tune. Understanding the difference is what keeps you from overpaying for the wrong thing.
Training is spent once, inference is spent forever
Here's the key economic fact: training is (mostly) a one-time cost. Once the weights are set, you can copy them and run the model as many times as you like. Every time you actually use the model, though, you pay again — in compute, latency, and (with cloud APIs) money. That running cost is the subject of the next page.
Common questions
How is an AI model trained?
Training shows a model enormous amounts of text and, at each step, has it predict the next token, compares the guess to the real next token, scores the error as a number called loss, and nudges every weight to reduce that loss. Repeat billions of times and the model becomes good at predicting real human text.
What is "loss" in machine learning?
Loss is a single number measuring how wrong the model's prediction was. A bad guess produces high loss; a good guess produces low loss. Training continuously adjusts the model's weights to push the loss lower — a falling loss is the model learning.
How much does it cost to train an AI model?
Training a large frontier model can cost millions of dollars, using thousands of specialized chips running for weeks on trillions of tokens. Most businesses never pretrain from scratch, though — they start from an existing model and, at most, fine-tune it, which costs far less.
What's the difference between pretraining and fine-tuning?
Pretraining builds a model from scratch on a massive general dataset — the expensive, multi-million-dollar part only a few labs do. Fine-tuning takes an already-trained model and nudges it on a small, focused dataset to specialize its style or domain, which is cheap and fast by comparison.