Technical

Overfitting

A condition where a model learns patterns specific to its training data that don't generalize to unseen data, resulting in high training accuracy but poor real-world performance.

Explained at five levels

Level 1

When an AI memorizes answers instead of truly learning — like acing a test by memorizing answers but not understanding the subject.

Level 2

When a model learns the training data too well, including its noise and quirks, so it performs great on practice data but poorly on new data.

Level 3

A condition where a model learns patterns specific to its training data that don't generalize to unseen data, resulting in high training accuracy but poor real-world performance.

Level 4

Excessive model complexity relative to training data, where the model captures noise and idiosyncrasies rather than underlying patterns — mitigated by regularization, dropout, early stopping, and data augmentation.

Level 5

A regime where empirical risk on the training set decreases while true risk increases — diagnosable via train-test divergence curves, addressable through capacity control (L1/L2 regularization, dropout, weight decay) and the double descent phenomenon in overparameterized models.

Definitions are educational summaries. Terminology can vary by source and context.

Sources