Architecture

Transformer

A neural network architecture that uses self-attention mechanisms to process sequential data in parallel, forming the foundation of most modern LLMs.

Explained at five levels

Level 1

The special design inside modern AI that lets it pay attention to all parts of a sentence at once — like reading a whole page instead of one word at a time.

Level 2

The type of AI architecture behind ChatGPT, Claude, and other modern AI. It's really good at understanding the relationships between words in a sentence.

Level 3

A neural network architecture that uses self-attention mechanisms to process sequential data in parallel, forming the foundation of most modern LLMs.

Level 4

The dominant sequence modeling architecture based on multi-head self-attention and position-wise feed-forward layers, enabling parallel computation and capturing long-range dependencies more effectively than RNNs.

Level 5

An architecture employing scaled dot-product attention over queries, keys, and values with multi-head projections, achieving O(n²d) complexity per layer — foundational to the scaling hypothesis and emergent capability literature.

Definitions are educational summaries. Terminology can vary by source and context.

Sources