Diffusion Training Objective
The pretraining-style regime where a denoiser learns to predict noise or a clean-direction signal from corrupted examples across a noise schedule.
The diffusion training objective teaches a model to reverse controlled corruption: training starts from clean examples, adds a known amount of noise at a chosen schedule level, and asks the model to predict the noise or the direction back toward a cleaner state. The model learns one denoising correction at a time rather than producing a finished sample in a single step.
At a glance
Released
June 2020
Authors
Jonathan Ho, Ajay Jain, Pieter Abbeel
Regime type
Training Pretraining
Related modules
No related modules listed yet.
What It Is
The diffusion training objective is the learning target used to train denoising generative models. Training pairs a clean example with a noised version of that example and asks the network to predict the corruption signal needed to undo one step of noise. Variants may target the added noise directly, a score that points toward higher data density, or an estimate of the clean sample, but the shared idea is the same: learn how to move one step from a noisy state toward a plausible clean state.Why It Exists
Many generative systems need a training recipe that matches how they will later produce outputs. Diffusion models do not emit a finished image, audio clip, or latent grid in one forward pass. They refine a noisy state through many small updates. The diffusion training objective exists so the model learns those partial corrections during training instead of being trained for a one-shot reconstruction task that does not match generation.How It Works
A training batch begins with clean data such as pixels or latents. A noise schedule picks how much corruption to add at each training level, producing a noised state at level t. The denoiser reads that noisy state and the level index, then outputs a prediction of the noise, score, or clean-direction signal needed to reverse one denoising step. The loss compares that prediction to the known corruption and updates the weights. Repeating this across many examples and schedule levels teaches the model behavior from nearly clean states through highly noisy ones. At generation time the same one-step correction is applied repeatedly in reverse along the schedule, because each update only partially denoises the state rather than producing the final sample in one prediction.Diffusion training objective flow
Clean training exampleNoise schedule adds corruption at level tDenoiser predicts noise or clean-directionRepeated reverse denoising at generationClean training example to Noise schedule adds corruption at level tNoise schedule adds corruption at level t to Denoiser predicts noise or clean-directionDenoiser predicts noise or clean-direction to Repeated reverse denoising at generation
Training and denoising flow
\\mathcal{L} = \\mathbb{E}_{t,\\,x_0,\\,\\epsilon}\\left[\\left\\lVert \\epsilon - \\epsilon_\\theta(x_t, t) \\right\\rVert^2\\right]