Training Data

Level 1

All the books, websites, and conversations the AI read to learn how to talk — like going to a really, really big school.

Level 2

The huge collection of text, images, or other data that an AI studied to learn. The better and bigger the training data, the smarter the AI.

Level 3

The dataset used to train a machine learning model. For LLMs, this typically includes web pages, books, code, and other text corpora totaling billions of tokens.

Level 4

The corpus of labeled or unlabeled examples used during the optimization of model parameters. Data quality, diversity, and scale directly impact model capabilities and biases.

Level 5

The empirical distribution D from which training examples are drawn, governing the model's inductive bias and generalization bounds — subject to distribution shift, label noise, memorization vs. compression tradeoffs, and data contamination risks.

Sources

• NIST Trustworthy and Responsible AI Resource Center: Glossary — National Institute of Standards and Technology. Accessed 2026-07-20.