We can't find the internet
Attempting to reconnect
Something went wrong!
Attempting to reconnect
← Generative Models step 8 of 11
Classifier-Free Guidance
Classifier-free guidance combines two noise predictions for the same noisy sample at the same diffusion timestep. One prediction uses conditioning, such as a text prompt; the other does not. The difference between them provides a direction for adjusting the prediction without a separate classifier.
Implement cfg_combine(eps_uncond, eps_cond, guidance_scale). The predictions are already supplied: this function only combines them. It does not run a model, draw random noise or perform a denoising step.
Inputs and output
- eps_uncond: unconditional noise predictions, shape (N, d).
- eps_cond: conditional noise predictions, also shape (N, d).
- guidance_scale: a finite, nonnegative Python float, shared by all elements.
The two tensors have the same floating-point dtype and device. Return a tensor with the same shape, dtype and device, leaving both inputs unchanged.
Combination rule
eps_guided = eps_uncond + guidance_scale * (eps_cond - eps_uncond)
Apply this elementwise. Do not average across the batch or feature axis, normalize the difference, or clamp the result.
| Scale | Result |
|---|---|
| 0 | The unconditional prediction |
| Between 0 and 1 | A point between the two predictions |
| 1 | The conditional prediction |
| Greater than 1 | A point beyond the conditional prediction in the same direction |
For eps_uncond=[[1, 4]] and eps_cond=[[3, 2]], the difference is [[2, -2]]. A scale of 0.5 gives [[2, 3]], while a scale of 2 gives [[5, 0]]. Notice that guidance can decrease an element as well as increase it. If the predictions are equal, the difference is zero and the result does not change with the scale.
A larger scale amplifies the conditioning difference. It is not a guarantee of better generated samples; guidance involves a quality-diversity tradeoff. There is no model-specific default scale in this exercise.
See Ho and Salimans, Classifier-Free Diffusion Guidance.
Understand the ideas behind this exercise. The Problem tab contains the exact implementation requirements.
Strengthen the effect of a prompt or other condition
Classifier-free guidance combines two predictions for the same noisy sample:
one made with a condition, such as a text prompt, and one without it. For noise
predictions epsilon_c and epsilon_u, a common convention is
$$\epsilon_{guided}=\epsilon_u+s(\epsilon_c-\epsilon_u).$$
The scale s determines the combination:
s = 0uses the unconditional prediction.s = 1uses the conditional prediction.s > 1moves beyond the conditional prediction in the same direction.
If the unconditional prediction is 2 and the conditional prediction is 5,
scale 2 gives 2 + 2 * (5 - 2) = 8. This is extrapolation, not an average
between 2 and 5. If the two predictions are equal, the result stays the same
at every scale.
Use matching predictions
The model must support both conditional and unconditional behavior. One common training method occasionally drops the condition. Combining the two predictions does not require a separate classifier or its gradient.
Both predictions must refer to the same noisy input and time step, and must represent the same quantity. For example, do not combine a noise prediction from one branch with a clean-sample prediction from the other using this formula.
Larger guidance can strengthen conditioning, but may reduce diversity or introduce artifacts. There is no universally best scale. Use the convention and value specified by the exercise.
Your code, saved here
Sign in to access your accepted solutions and saved draft history. You can keep reading the lesson without an account.
Sign in to your accountStuck?
JAX reference solution
Sign in to track your attempts toward unlocking the reference solution.