Taxonomy

How attention, feed-forward layers, normalization, residuals, and position information repeat inside each decoder-style transformer block.

Post-training steps that steer a model toward helpful, honest, and policy-compliant behavior—often via preference data, reward models, or safety filters.

The blueprint that defines how a model's layers, modules, and data flow connect during inference.

A generation paradigm that produces outputs one discrete step at a time, each step conditioned on everything generated so far.

A finer-grained part inside a module, such as an attention head or projection layer.

Extra inputs—text prompts, class labels, images, or guidance weights—that steer a generative model without retraining the full parameter stack each time.

A network stack that turns internal representations into outputs—tokens, pixels, or structured predictions—often one step at a time with attention to prior context.

A generation paradigm that starts from noisy latents and iteratively removes noise—or adds structure—until a clean sample emerges.

A generative model family that learns to reverse a noise corruption process through many iterative denoising steps.

A model trained to score, rank, or classify inputs into labels or preferences rather than synthesize new content.

Qualitative capability jumps that appear only past certain model scale or training thresholds—distinct from smooth metric improvements tracked by scaling laws.

A network stack that maps raw or patched inputs into internal representations—tokens, latents, or context vectors—for downstream decoders or heads.

An architecture that compresses input into representations with an encoder, then generates outputs with a decoder—often via cross-attention between the two stacks.

A large pretrained model intended as a shared starting point for many downstream tasks via fine-tuning or prompting.

How well a model performs on data it was not explicitly trained to memorize—validation sets, new domains, and live traffic are the practical tests.

A model trained to produce new outputs such as text, images, or audio rather than only assign labels to fixed categories.

A compressed internal code that summarizes input structure for reconstruction, generation, or downstream tasks.

The coordinate system in which latent codes live, where distance and direction correspond to semantic or structural variation in the data.

The kind of input or output data a model is built to handle, such as text, images, audio, or video.

A trained machine learning system that maps inputs to outputs using learned parameters.

How much function a model can represent—driven by parameter count, width, depth, and training data—before optimization and regularization constrain what it actually learns.

A reusable building block inside a model architecture, such as attention or a feed-forward network.

A model family that accepts or produces more than one data modality, such as text paired with images or audio.

When a model fits training noise so closely that held-out or production data perform worse—the classic gap between memorization and useful generalization.

A fixed-size chunk of input data—pixels, spectrogram frames, or grouped tokens—that an encoder turns into a representation vector.

A standard language-model evaluation metric: exponentiated average cross-entropy over tokens, lower is better when comparing models on the same corpus.

An internal encoding of input data that downstream layers or tasks use instead of raw features.

Empirical power-law relationships between model scale, compute, data, and downstream loss or capability—used to forecast training budgets and benchmark trends.

A model family built from stacked attention and feed-forward blocks that mix information across token positions in parallel.

A model family that learns environment dynamics or state transitions so it can predict or simulate what happens next.