Glossary
Latent
A compressed internal code that summarizes input structure for reconstruction, generation, or downstream tasks.
What It Is
Latents appear after encoders or inside generative objectives: VAE means, diffusion x_t states at a timestep, or bottleneck vectors in autoencoders. They are representations chosen for efficiency or controllability, not raw patches or tokenizer outputs.
Why It Matters
Separating latent from patch and from the full latent space helps you compare diffusion, VAE, and autoregressive stacks. It also clarifies which tensors are safe to manipulate at inference—interpolating latents, swapping style codes, or conditioning denoisers.
Simple Example
An image autoencoder maps a photo to a 512-dimensional latent vector, then a decoder reconstructs pixels from that vector. The latent is the bottleneck code; the decoder readout is not itself the latent.
Common Confusions
Latent versus representation: every latent is a representation, but not every hidden state is called a latent—authors usually reserve the term for compressed or stochastic codes. Latent versus embedding: embeddings often mean discrete lookup tables at the input; latents are typically continuous codes deeper in the stack. Latent is not the same as a noise sample in diffusion unless the text explicitly refers to the noised state as the working variable.