Machine learning, from the inside
Learn how language models work, how to run one, and how to tell if it works well.
A language model learns patterns from examples, then uses those patterns to produce text. This guide explains what happens inside the model and what it takes to run one well. No machine learning background is required.
The examples let you explore the ideas by changing a few settings. Their numbers illustrate how things work; they are not performance promises.
In this guide
Tokens and contextHow text becomes numbers, and why the context window is a hard limit.
TransformersAttention, layers, and the KV cache that grows with every token.
TrainingLoss, gradients, learning rates, and overfitting.
Fine-tuningLoRA, supervised tuning, and preference tuning.
InferencePrefill and decode, TTFT, and why a chat request has two phases.
The memory wallHow memory traffic can limit decode speed.
QuantizationBit width, groups, FP8, NVFP4, EXL3, and the quality cost.
KernelsFusion, tiling, coalescing, and what a profile can tell you.
Mixture of expertsSparse compute, expert routing, and load imbalance.
Multi-token predictionDraft heads, acceptance rates, and speculative speedup.
Serving systemsContinuous batching, prefix caching, and queue limits.
MeasurementHow to tell a real speedup from noise.
Image and video modelsDiffusion steps, guidance, distillation, and cost scaling.
How-to recipesTask steps for sizing, quantization, benchmarking, and limits.
Mixed reviewTwenty mixed questions with hidden answers.
GlossaryShort definitions for the terms in this guide.