Core
Scaling Laws
Empirical relationships showing that model performance improves predictably as a power law of compute, parameters, and data, guiding resource allocation for AI training.
Explained at five levels
Level 1
The rule that says bigger AI with more data almost always gets smarter — like how eating more books makes you know more stuff.
Level 2
The discovery that AI performance improves predictably as you increase model size, training data, and computing power. This is why companies keep building bigger models.
Level 3
Empirical relationships showing that model performance improves predictably as a power law of compute, parameters, and data, guiding resource allocation for AI training.
Level 4
Power-law relationships between model loss and scaling dimensions (parameters N, dataset size D, compute C) discovered by Kaplan et al. and refined by Chinchilla, establishing that optimal training balances model size with data.
Level 5
Empirical power-law scaling L(N,D,C) ∝ N^(-α) + D^(-β) + C^(-γ) governing the compute-optimal frontier — with implications for training budget allocation, capability prediction, and the extrapolability of benchmark performance to larger scales.
Definitions are educational summaries. Terminology can vary by source and context.
Sources
- NIST Trustworthy and Responsible AI Resource Center: Glossary — National Institute of Standards and Technology. Accessed 2026-07-20.