We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
← Generative Models step 5 of 11
DDPM Noise Schedule
A diffusion noise schedule specifies the variance added at each step. Implement linear_beta_schedule(num_steps, beta_start=0.0001, beta_end=0.02) to build a linear schedule and its cumulative alpha values.
Inputs and output
num_steps is a positive integer T. The endpoints are finite floats with 0 <= beta_start <= beta_end < 1.
Return a float32 tensor of shape (2, T):
- Row 0 contains the beta values.
- Row 1 contains alpha_bar, the cumulative product of 1 - beta.
For T greater than 1, create T evenly spaced beta values including both endpoints. For T=1, use only beta_start, even if beta_end differs.
Cumulative products and indexing
Use zero-based array indices:
alphas[t] = 1 - betas[t]
alpha_bar[t] = alphas[0] * alphas[1] * ... * alphas[t]
The product includes index t. In particular, alpha_bar[0] is 1 - beta_start, not 1: index 0 represents the first noising step. Index t therefore describes t + 1 steps.
For T=3, beta_start=0.1 and beta_end=0.3, return these rows:
betas: [0.1, 0.2, 0.3]
alpha_bar: [0.9, 0.72, 0.504]
The forward-noising exercise uses sqrt(alpha_bar[t]) to scale the clean input and sqrt(1 - alpha_bar[t]) to scale standard Gaussian noise. alpha_bar itself is not the multiplier applied directly to the clean input.
The final cumulative value depends on both endpoints and the number of steps. A short schedule need not end near zero. If every beta is zero, every cumulative alpha is one.
See Denoising Diffusion Probabilistic Models by Ho, Jain and Abbeel.
Understand the ideas behind this exercise. The Problem tab contains the exact implementation requirements.
Train a model to recognize added noise
A diffusion model learns from data with different amounts of added noise. Let
beta_t specify the noise schedule, alpha_t = 1 - beta_t, and alpha_bar_t
be the product of the alpha values up to step t. A noisy sample can be drawn
from clean data x_0 directly:
$$x_t=\sqrt{\bar\alpha_t}x_0+\sqrt{1-\bar\alpha_t}\epsilon, \qquad\epsilon\sim\mathcal N(0,I).$$
If alpha_bar_t = 0.64, the coefficients are 0.8 for the clean input and 0.6
for the noise. These are amplitude coefficients; their squares determine the
corresponding variance factors. When alpha_bar_t is near one, much of the clean
signal remains. Near zero, noise dominates. The chosen schedule determines
whether the final training step reaches nearly pure noise.
For noise-prediction training, compare the model's output with the actual
epsilon used to construct x_t. Also supply the time step so the model knows
the noise level. A model that predicts the clean sample instead needs different
targets and sampling formulas.
Generate by taking reverse steps
Generation usually starts from noise. At each step, the model helps compute a
less noisy sample. A stochastic sampler can also add fresh noise, using the
variance specified by its schedule. Using beta_t as that variance is different
from using the forward process's posterior variance. The final step commonly
omits added noise.
Keep array indices separate from mathematical time-step labels. At array index zero, a cumulative product already includes the first factor. Follow the task's random-seed or key sequence too: changing the order of draws can change expected outputs. A small linear model can demonstrate these mechanics without being a realistic image generator.
Your code, saved here
Sign in to access your accepted solutions and saved draft history. You can keep reading the lesson without an account.
Sign in to your accountStuck?
JAX reference solution
Sign in to track your attempts toward unlocking the reference solution.