1.4 Part 1 · Imitation learning · foundations

Diffusion models

Regression outputs the average of the answers it was shown. When there are two correct answers, the average is wrong: the robot drives between the two detours straight into the kettle. We need a model that outputs one of the correct answers. This lesson takes apart such a model, the diffusion model. It learns to remove noise from data and generates the other way round: it starts from pure noise and step by step moves toward the data. To see how this works, the labs in this lesson use points on a plane instead of robot actions: a set of points laid out in two spirals is first blurred with random noise, then a network is trained to guess which noise was added, and finally the path is walked backwards, from random points to the spirals. The same scheme works with robot actions: at the end of the lesson you'll check it on the kettle detour from lesson 1.3. If you already know diffusion models, you can skip this lesson.

≈ 45 minutes1 lab and 6 interactives16 missions with a goal

NOISEDraw how the signal fades step by step and compare it with the real curve

DENOISERScrub through a generation and find when points are pulled to the center and when they pick an arm

LABTrain a network in your browser, get spirals out of noise, and compare two ways of generating on the same network: with fresh noise at every step and without it

ON REQUESTAsk for points on a chosen spiral, build the precision-diversity curve, and tune how hard to push the request

MEMORIZATIONFind when a small network starts copying the training points

ADA'S ACTIONSTrain the same denoiser on kettle-detour actions and check that none of the 100 leads into the kettle

Start the lesson ↓

1 Generator

A generator instead of a single answer

A policy trained with mean squared error outputs the average of the actions it was shown. If the operator went around the kettle sometimes on the left and sometimes on the right, the average of the two detours leads straight into the kettle. That's how the loss itself works: its minimum is the mean of all correct answers, and more data doesn't change that.

In lesson 1.3 we got around the mean in two ways. A mixture of Gaussians picks one of several options, but their number has to be known in advance, and each option is a simple blob. Bins split each coordinate into intervals, but independent bins lose the link between coordinates, and joint bins for many coordinates are far too many.

So we need a model that outputs a sample: a random example from a data distribution of any shape. Such a model is called a generator. Run it twice and you get two different answers, both of which look like the data: one detour on the left, the other on the right.

Let's take the generator apart on the simplest data: points on a plane. They lie on two spirals, 1000 points on each. The spirals play the role of the two detours: a good generator puts points on the spirals, equally on both, and not between the arms. The lesson's reference papers are DDPM (Ho, Jain, Abbeel, 2020) and DDIM (Song, Meng, Ermon, 2021).

Random noise is easy to get: it's a standard function in any library. How do you turn noise into a point on a spiral?

2 Forward process

The path from data to noise

Turning noise into a point on a spiral in one step is hard: a network trained for that transformation with mean squared error outputs the mean again. You'll see this for yourself in the lab.

A diffusion model gets around it like this. First the data is deliberately corrupted: random noise is added to the spiral points step by step until nothing is left of the spirals but a shapeless cloud. This path is fixed in advance, and at every step we know exactly which noise was added. That is what the network is trained on: it is shown a noised point and the step number, and it guesses the noise that was added. Once the network can do that, the path is walked backwards: take a new random cloud, at every step the network predicts the noise, part of it is removed, and after many steps spirals emerge from the cloud. These are new points that weren't in the data.

Each square is a view of the plane with points, same axes as in the lesson's labs. Top row (left column on a phone), from data to noise: 220 points of two spirals and the same points after 250, 500, 750 and 1000 steps of adding noise. Bottom row (right column), from noise to data: a new random cloud of 220 points and the same cloud after the noise has been removed down to step 750, 500, 250 and 0. The frame on a square marks the current step. The rectangle in the middle is the denoiser network; the dashed line to it from the top row is training, from it to the bottom row is generation. The snapshots of the reverse path were produced by the ideal denoiser from section 3; the network trained in the lab gives the same. Click the diagram to pause it.

The top row of the diagram is this section: how the path from data to noise works and how much signal remains at each step. The bottom row is section 3: how the network knows where to move a point on the reverse path. In the lab you'll train such a network and walk the reverse path yourself.

Let's start with the path from data to noise. It's simple: at every step the point is shrunk slightly toward the center, and a little Gaussian noise is added to it. After $T = 1000$ steps the spirals have turned into a Gaussian cloud. This path is fixed; there's nothing to learn. How much noise to add at step $t$ is set by the schedule. In DDPM it is linear: the portion of noise added in one step grows from 0.0001 at the first step to 0.02 at the last.

It's convenient to track the process with a single number, $\bar\alpha_t$ , the fraction of the signal that remains of the original point after all steps up to $t$ . This is not the portion of a single step but the total over all steps at once. At the start $\bar\alpha = 1$ , the point is clean; by the end $\bar\alpha$ is almost zero. Any point can be noised straight to any step without passing through the intermediate ones.

First draw how you think $\bar\alpha_t$ falls under the linear schedule; the cloud on the left will hint at how much signal is left at step $t$ . Then compare your prediction with the real curve and with the cosine schedule from Improved DDPM (Nichol, Dhariwal, 2021).

Step t0

Under the hood: the forward process and two schedules

A forward-process step in DDPM notation:

\begin{aligned} q(x_t \mid x_{t-1}) &= \mathcal{N}\bigl(\sqrt{1 - \beta_t}\,x_{t-1},\ \beta_t I\bigr),\\ \alpha_t &= 1 - \beta_t, \quad \bar\alpha_t = \alpha_1 \cdots \alpha_t,\\ x_t &= \sqrt{\bar\alpha_t}\,x_0 + \sqrt{1 - \bar\alpha_t}\,\varepsilon,\\ \varepsilon &\sim \mathcal{N}(0, I). \end{aligned}

The last line is what lets you noise a point straight to step $t$ . The DDPM linear schedule: $\beta$ grows from $10^{-4}$ to 0.02 over $T = 1000$ steps. Hence $\bar\alpha_{500} \approx 0.08$ and $\bar\alpha_T \approx 4 \cdot 10^{-5}$ ; this is our computation from the paper's formulas. The DDPM authors themselves estimate that $x_T$ differs from Gaussian noise by $\mathrm{KL} \approx 10^{-5}$ bits per dimension.

The cosine schedule from Improved DDPM is defined directly through $\bar\alpha$ :

\begin{gathered} \bar\alpha_t = \frac{f(t)}{f(0)}, \qquad s = 0.008,\\ f(t) = \cos^2\Bigl(\frac{t/T + s}{1 + s}\cdot\frac{\pi}{2}\Bigr). \end{gathered}

From it $\beta_t = 1 - \bar\alpha_t / \bar\alpha_{t-1}$ , clipped from above at 0.999. Under this schedule $\bar\alpha_{500} \approx 0.49$ .

How the prediction error is computed. The prediction is stored at 41 points, every 25 steps. The error is the mean vertical distance between the prediction and the real curve of the linear schedule at those points.

So the linear schedule loses signal fast: by the middle of the path less than a tenth is left, and over the last third of the path the points are almost indistinguishable from noise. We know the path from data to noise exactly. How do we walk it backwards, from noise to spirals?

3 Denoiser

Where to move a point on the reverse path

To walk the path backwards, at every step you need to know where to move the noised point so that it looks more like the data. That's hard: a noised point could have come from many places, and the stronger the noise, the more candidates there are. If at every step you move it toward the mean of all candidates, it ends up in the center, between the spirals. That's the same error regression makes.

The way out is to ask the denoiser at every step: it estimates where the noised point came from. If you know all the training points, the best estimate is their weighted average. The closer a training point, the larger its weight, and how fast the weight falls off with distance depends on the noise level. The arrow from the point to the estimate shows the direction toward the data. The field of such arrows is called the score; in the EDM paper (Karras et al., 2022) it is "a vector field that points towards higher density of data at a given noise level".

On the scene, a generation driven by the ideal denoiser: it knows all 2000 points, there's no network here. Generation here takes 50 steps rather than 1000: each step jumps over several values of $t$ ; how that works is covered in the lab. Scrub through the generation step by step. The orange points are 300 samples at this step; the arrows show where the denoiser pulls the points on the scene. You can drag the probe point: the cross is its estimate, and the training points the denoiser averages are highlighted.

Gray points are the 2000 training points of the two spirals; the "mean" cross is their overall mean. The line at the top of the scene: the generation step number, the forward-process step $t$ , and the signal fraction $\bar\alpha$ at it.

Generation step0

Under the hood: the ideal denoiser and the score

In EDM notation a point is noised as $x = y + n$ , $n \sim \mathcal{N}(0, \sigma^2 I)$ . For a finite sample $y_1, \ldots, y_N$ the optimal denoiser and the score have a closed form:

\begin{aligned} D(x; \sigma) &= \frac{\sum_i \mathcal{N}(x;\, y_i, \sigma^2 I)\, y_i}{\sum_i \mathcal{N}(x;\, y_i, \sigma^2 I)},\\ \nabla_x \log p(x; \sigma) &= \frac{D(x; \sigma) - x}{\sigma^2}. \end{aligned}

DDPM step $t$ corresponds to noise level $\sigma = \sqrt{(1 - \bar\alpha_t)/\bar\alpha_t}$ if $x_t$ is divided by $\sqrt{\bar\alpha_t}$ . At the end of the path $\sigma = 0.01$ , at the start about 160. In DDPM notation the score is tied to the predicted noise:

\nabla_{x_t} \log q_t(x_t) \approx -\frac{\varepsilon_\theta(x_t, t)}{\sqrt{1 - \bar\alpha_t}}.

So a network that predicts the noise also knows the direction toward the data. The score is the gradient with respect to the input $x$ , not the network parameters.

The output $D(x; \sigma)$ is a weighted average of the training points, so such a denoiser can only reproduce the data (EDM, appendix B.3, eq. 57; Gu et al.). We'll come back to this in the section on memorization.

How the scene works. Generation is DDIM over 50 steps, 300 points from the same noise, step times $t_i = \operatorname{round}\bigl(1000\,(i/50)^2\bigr)$ . The arrows sit at the nodes of a 15 × 15 grid with spacing 0.3 and show the direction from the node to the denoiser's estimate at the current step. An arrow points to the mean if it deviates from the direction to the data's center of mass by less than 15°; nodes closer than 0.3 to the center are not counted. An arrow leads to the nearest spiral if it deviates from the direction to the nearest data point by less than 30°; we count nodes inside the circle of radius 1.75 where the spirals lie, and no closer than 0.08 to the data.

So at the start of generation the denoiser pulls all points to the center, at the end to the nearest arm, and between these stages a point chooses its spiral. But the ideal denoiser stores all 2000 points and goes through all of them at every step. Can we get the same arrows from a network that doesn't store the data?

🔬 Lab

Diffusion on two spirals

We can: the denoiser is replaced by a network that learns from examples and doesn't store the data. For images and robot actions there's no other way: there are millions of examples, and going through them at every step takes far too long. Training runs in your browser and takes seconds.

The network is trained like this: take a spiral point, a random step $t$ and random noise, noise the point straight to step $t$ , and ask the network to guess the noise that was added. The loss is the mean squared difference between the true and the predicted noise. The DDPM paper calls it $L_{\text{simple}}$ . Predicting noise is convenient: it is always on the same scale, and from the noise it's easy to compute the estimate of the clean point, that is, the same arrow as in section 3.

Generation runs the other way: take a point of the Gaussian cloud and go from $t = 1000$ to zero. At every step the network predicts the noise, we remove part of it, and the point moves one step closer to the data. DDPM also adds a little fresh noise at every step, so the path is random. The fresh noise isn't there by accident: after part of the noise is removed, the point is cleaner than it should be at the next step, while the network was trained on points with the proper noise level, and the fresh portion brings the point back to that level. It also provides spread: from one start you get different answers, the generated points cover both spirals entirely rather than collapsing onto a few spots, and the network's error from the previous step is partly smoothed out. DDIM takes the same trained network but steps without fresh noise, and the path from start to finish is deterministic. The $\eta$ slider in the lab blends the two methods: $\eta = 1$ is DDPM, $\eta = 0$ is DDIM. In the diagram both methods start from the same noise point with the same denoiser.

Both panels are a view of the plane with points, as in the lab: gray are 500 of the 2000 training points of the two spirals. The black circle is the same starting noise point; the denoiser on both panels is the same, the ideal one from section 3. On the left, 3 DDPM runs of 50 steps each: at every step fresh random noise is added to the denoiser's shift, the paths jitter (older segments are paler), and from one start you get 3 different endpoints. On the right, DDIM from the same start: without fresh noise the path is smooth and the same on repeat; the orange line is 50 steps, the purple one with nodes is 10 steps, and the endpoint is almost the same, so you can take few steps. Both numbers are on the "Generation steps" slider in the lab. Click the diagram to pause it.

In the lab you'll train the network and compare it with a "noise → point" regression. Training runs in epochs: an epoch is one pass over all 2000 training points. Generation always starts from 1000 random points of Gaussian noise, and the network's job is to remove the noise from them in the chosen number of steps so that the points land on the spirals. You'll find how many steps DDPM and DDIM need, and the boundary beyond which neighboring starting points go to different spirals.

DDPMnetwork is training

The scene is a view of the plane with points, as in sections 2 and 3: gray points are the 2000 training points of the two spirals, spiral 2 is lighter. Generation starts from 1000 random points of Gaussian noise, in orange; at every step the network removes part of the noise from them, and the thin lines are the paths of 40 of them. Generation succeeded if the points landed on the spirals. The line at the top of the panel shows what's on the scene right now: training and the number of epochs, or the sampler and the number of steps. The plot below is the network's loss by epoch; the dashed line is the mission threshold.

Epochs per click100 Generation steps100 DDIM alongside η of left panel1.0 regression alongside basin map

Under the hood: the network and training

\begin{gathered} L_{\text{simple}} = \E\,\bigl\lVert \varepsilon - \varepsilon_\theta(x_t, t)\bigr\rVert^2,\\ x_t = \sqrt{\bar\alpha_t}\,x_0 + \sqrt{1 - \bar\alpha_t}\,\varepsilon. \end{gathered}

Input. The point's coordinates, 16 sine-cosine pairs of random projections of the coordinates (Fourier features: with them the network picks up fine detail faster), and an embedding of the step number: $t/T$ and sines and cosines of $t$ at five frequencies. 45 numbers in total.
Network. Two hidden layers of 64 neurons with SiLU activation, two numbers at the output: the predicted noise. About 7300 parameters. For comparison, DDPM for CIFAR-10 has 35.7 million.
Training. Adam, batch of 125 examples, an epoch is a pass over 2000 points, that is, 16 steps. Learning rate $3 \cdot 10^{-3}$ , after 200 epochs $10^{-3}$ , after 400 $5 \cdot 10^{-4}$ .
Departure from DDPM. In DDPM the step number $t$ during training is uniform. We pick small $t$ more often: $t = 1 + \lfloor T u^2 \rfloor$ , where $u$ is uniform from 0 to 1. The fine structure of the spirals is visible only at low noise, and this way the network learns several times faster.
Loss curve. This is $L_{\text{simple}}$ on a fixed set of 4000 triples $(x_0, t, \varepsilon)$ with uniform $t$ , as in the paper. We average the loss over the two coordinates.
"Noise → point" regression. The same network without the step number. 400 Adam steps on MSE: random noise at the input, a random data point at the output.

Under the hood: DDPM and DDIM steps

A DDPM step, $z \sim \mathcal{N}(0, I)$ :

x_{t-1} = \frac{1}{\sqrt{\alpha_t}}\Bigl(x_t - \frac{\beta_t}{\sqrt{1 - \bar\alpha_t}}\,\varepsilon_\theta(x_t, t)\Bigr) + \sigma_t z.

A DDIM step from time $t$ to an earlier time $t'$ :

\begin{aligned} \hat x_0 &= \frac{x_t - \sqrt{1 - \bar\alpha_t}\,\varepsilon_\theta}{\sqrt{\bar\alpha_t}},\\ x_{t'} &= \sqrt{\bar\alpha_{t'}}\,\hat x_0 + \sqrt{1 - \bar\alpha_{t'} - \sigma^2}\,\varepsilon_\theta + \sigma\varepsilon,\\ \sigma &= \eta\,\sqrt{\frac{1 - \bar\alpha_{t'}}{1 - \bar\alpha_t}}\,\sqrt{1 - \frac{\bar\alpha_t}{\bar\alpha_{t'}}}. \end{aligned}

At $\eta = 0$ the step is deterministic; at $\eta = 1$ the sampler coincides with DDPM. Watch the notation: in the DDIM paper the letter $\alpha$ denotes what DDPM calls $\bar\alpha$ .

How the lab's samplers work. Both are the same DDIM formula; DDPM is the case $\eta = 1$ . The network is always the same one, trained on all 1000 steps. If the number of generation steps $S$ is less than 1000, we take the times $t_i = \operatorname{round}\bigl(T\,(i/S)^2\bigr)$ : near the end of the path, where the noise is small, steps are denser. The last step always outputs the estimate $\hat x_0$ without noise. We keep the estimate $\hat x_0$ inside the square from −2.2 to 2.2, the way pixels are clipped for images.

Under the hood: how the criteria are computed

A point is on a spiral if the nearest of the 2000 training points is closer than 0.1. That point's spiral counts as the point's spiral. The spiral width in the data is noise with standard deviation 0.05; the distance between neighboring arms is about 0.27.
Outside the circle. The dashed circle of radius 1.14 splits the data in half: exactly half of the training points lie outside it. If the sampler pulls the cloud to the center, fewer points end up outside the circle.
The shape holds if at least 80% of the points are on the spirals and at least 25% are outside the circle.
Generation. 1000 points from the same starting noise; DDIM alongside uses the same noise. Trajectories are drawn for 40 points. In the basin mission: DDIM over 20 steps.
Basin map. From the nodes of a 41 × 41 grid we run DDIM over 20 steps and color the node by the spiral where the endpoint landed. Nodes whose endpoint didn't land on a spiral are left uncolored.

So the network replaced the ideal denoiser, and generation puts points on the spirals, with the start deciding which one. DDIM on the same network holds the shape at 6–8 steps, where DDPM already pulls the cloud to the center. A robot, though, needs an answer within tens of milliseconds. How many steps do real robots take, and how do they cut that number?

4 Steps in robots

How many steps robots take

Every generation step is a full pass of the network. DDPM takes 1000 steps, while a robot needs an answer within tens of milliseconds.

Simply taking fewer DDPM steps doesn't work. In the lab, at 6–8 steps the cloud collapsed to the center and the outer arms emptied: on a large step DDPM replaces the predicted noise with fresh noise and loses its sense of direction.

So the steps are taken in a way that keeps what has already been predicted. The simplest way is DDIM on the same network, as in the lab. Robot models take from 1 to 20 steps, most often 4 to 16: Diffusion Policy (Chi et al., 2023; lesson 1.5) on real robots takes 10 DDIM steps, π0 (the Physical Intelligence policy, 2024; lesson 3.6) takes 10 flow matching steps. Flow matching is a close relative of diffusion: it learns nearly straight paths from noise to data, so fewer steps are needed. We'll take it apart in lesson 1.8. The table by model and other ways to cut steps are in the block below.

Under the hood: how many steps robot and image models take

Besides DDIM, steps are cut in two more ways. Solvers like DPM-Solver step more accurately along the same path, so they need fewer steps. Distillation trains a second, fast network to repeat in 1–3 steps what the original does in many.

The table is a reference for later: most of the models in it are covered in the following lessons; the lesson number is in parentheses.

Model	Method	Inference steps
Diffusion Policy, 2023 — a diffusion policy on real robots (lesson 1.5)	DDIM, trained on 100 steps	10; 0.1 s on an RTX 3080
UMI, 2024 — a diffusion policy on data from a handheld gripper (lesson 4.2)	DDIM	16
Consistency Policy, 2024 — a distilled diffusion policy	distillation	1 or 3; 21 ms vs 192 ms for DDIM at 15 steps
Octo, 2024 — an open transformer policy with a diffusion head (lesson 3.5)	DDPM	20
RDT-1B, 2025 — a bimanual diffusion policy with 1.2 billion parameters	DPM-Solver++	5 instead of 100
π0, 2024 — the Physical Intelligence model (lesson 3.6)	flow matching	10; 73 ms on an RTX 4090
GR00T N1, 2025 — NVIDIA's model for humanoids (lesson 3.7)	flow matching	4; 63.9 ms on an L40
π0.7, 2026 — a later Physical Intelligence model (lesson 3.8)	flow matching	5

Careful: the configs in the Diffusion Policy repository default to 100 steps, not 10.

For images the order of magnitude is the same. CIFAR-10 is a standard set of small 32 × 32 images; generation quality on it is measured by FID: the lower the FID, the closer the images are to real ones. DDPM spends 17 seconds on it per batch of 256 images. DDIM at 50 steps gives FID 4.67 versus 4.04 at 1000, DPM-Solver 4.70 in 10 network calls, class-conditional EDM 1.79 in 35, consistency models 3.55 in a single step.

Flow matching with Gaussian noise at the input is equivalent to diffusion: the DDIM sampler coincides with the Euler method for flow matching, that is, with the simplest way of walking the path in short straight steps. And rectified flow learns straight paths from noise to data, so even fewer steps are needed. That's lesson 1.8.

Under the hood: a bridge to flow matching

Throughout the course, time in flow matching runs from noise to data: $t = 0$ is noise $x_0$ , $t = 1$ is data $x_1$ , as in the π0 paper. A point on the path and the training objective are written as:

\begin{aligned} x_t &= (1 - t)\,x_0 + t\,x_1,\\ L &= \E\,\bigl\lVert v_\theta(x_t, t) - (x_1 - x_0)\bigr\rVert^2. \end{aligned}

The network learns the velocity $v_\theta$ , and generation is motion along it from $t = 0$ to $t = 1$ . Note that in DDPM time runs the other way: $t = 0$ is data, $t = T$ is noise. In openpi, the open code of π0, noise also sits at $t = 1$ : x_t = t * noise + (1 - t) * actions, and integration uses the step dt = -1/10. Details in lesson 1.8.

The generator has become fast, but so far it outputs any plausible point. A robot needs an action for a specific situation. How do you make the generator output what was requested?

5 Condition and guidance

Generation on request

The generator outputs any data point, but a robot doesn't need just any: the action has to fit what the camera sees. In our example the request is the spiral number: "give me a point on spiral 1".

The simplest option is to feed the condition to the network's input: during training the network sees the spiral number of every point. This works only partly: without amplification, some points land on the wrong spiral or between the arms. How many, you'll check with a bet in the experiment below.

Amplifying the condition without a separate classifier is what classifier-free guidance does (Ho, Salimans, 2022). One network is trained for two tasks at once: during training the condition is sometimes dropped, with probability 0.1–0.2, and the network learns to predict noise both with and without the condition. At generation, both predictions are taken and the answer is pushed further away from the unconditional one. How much further is set by the weight $w$ .

The diagram below shows this on a single noised point. The gray arrow is where the network pulls it without the condition, the orange one with the condition "spiral 1". The result is an arrow from the same start toward the conditional one and beyond it: at w = 0 it coincides with the conditional prediction, at w = 1 it goes twice as far from the unconditional one, at w = 4 five times as far. On the right is what this produces on 500 points: as w grows they land on spiral 1 more confidently but more tightly, and at large w some overshoot the spiral.

On the left is a zoomed-in fragment of the plane with points: gray are training points, spiral 1 is darker; the circle is one noised point at step t = 175 of 1000; the network is the same as in the experiment below, after training. The gray arrow is its estimate of the clean point without a condition, the orange one with the condition "spiral 1", the thick dark one is the result at weight w, which cycles through 0, 1, 2, 4; the dashed line is the line along which the result shifts, and the ticks on it are the result's endpoints at each w. On the right is the whole plane: gray are 500 of the 2000 training points, orange are the same 500 generated points you'll get in the experiment at this w (DDIM, 20 steps, start from the same noise): at w = 1 precision is 92.6%, coverage 88.7%; at w = 4 they are 87.2% and 47.3%, and some points go past the spiral. While the network is training, the diagram shows the training progress. Click the diagram to pause it.

The network for the experiment is the same as in the lab, plus a "spiral number" input. Its training starts when the section appears on screen and takes a few seconds. Each generation puts a point on the plot: vertically precision, the fraction of points on the requested spiral; horizontally coverage, the fraction of that spiral's points that have at least one generated point next to them.

Weight w0

Under the hood: classifier-free guidance

\tilde\varepsilon = (1 + w)\,\varepsilon_\theta(x_t, c) - w\,\varepsilon_\theta(x_t, \varnothing)

Here $c$ is the condition and $\varnothing$ the empty condition. At $w = 0$ this is an ordinary conditional model. Papers write the weight differently: if the formula is written as $\varepsilon(\varnothing) + s\,\bigl(\varepsilon(c) - \varepsilon(\varnothing)\bigr)$ , then $s = 1 + w$ .

In the experiment: the same network as in the lab, plus two "spiral number" inputs; if the condition is dropped, both are zero. The probability of dropping the condition is 0.2; for Ho and Salimans the values 0.1 and 0.2 work equally well. 3000 Adam steps. Generation is DDIM over 20 steps, 500 points from the same noise. Precision is the fraction of generated points whose nearest training point is closer than 0.1 and lies on the requested spiral. Coverage is the fraction of the requested spiral's points that have a generated point closer than 0.1.

Real-Time Chunking (stitching action chunks, Physical Intelligence, 2025; lesson 1.5) also uses guidance, but a different kind: at every step it adds a gradient term to the velocity field so that the new action chunk agrees with the one already being executed.

It turns out that guidance makes generation more precise but poorer, and at a large weight the points crowd into a few spots on the spiral. The extreme case of poverty is a model that only repeats the training examples. Does that happen with diffusion?

6 Deeper

Memorization or generalization

It does. A generator is useful when it creates new examples that resemble the data, while a model that only repeats the training examples is a lookup table.

Why does this happen? The ideal denoiser from section 3 estimates the clean point as a weighted average of the training points. If you run a generation with it all the way to the end, every point lands exactly on one of the training points. And the network is trained to approximate precisely that.

What saves us is that the network can't memorize everything and doesn't have time to. While it has trained only briefly, it learns the smooth shape of the distribution and puts points between the training ones. On a small sample and with long training, the network gradually switches to copying.

To the left of the slider is the ideal denoiser, to the right a small network, both on the same $N$ points from the spirals. We count a sample as a copy if it's closer than 0.02 to one of the training points. Drag the slider, change N and the training step.

Training step8000

Under the hood: how the copy fraction is computed

The network is the same as in the lab; it trains for 8000 steps on batches of 64 examples. Every 1000 steps we generate 300 points with DDIM over 50 steps from the same noise and count what fraction of them is closer than 0.02 to a training point. The ideal denoiser generates with the same sampler. If we took fresh points from the spirals instead of samples, fewer than 6% would count as "copies" by chance even at $N = 128$ : that's how we checked that the 0.02 threshold doesn't confuse new points with copies.

For large models the following is known. Carlini et al. found 109 copies of training images among 175 million Stable Diffusion generations, and extracted 1280 images from CIFAR-10 models, 2.5% of the dataset. The main factor is duplicates in the data, and diffusion memorizes more than GANs. Kadkhodaie et al. showed that at $N \approx 10^5$ two networks trained on disjoint data generate almost identical images. Bonnaire et al.: the time to generalization doesn't depend on the sample size, while the time to memorization grows linearly with it, so there is a window for early stopping.

So the ideal denoiser only copies, and the network copies on small data and with long training. For a robot this means that on a dozen demonstrations a diffusion policy may replay the recorded motions instead of generalizing. And if we give the same network robot actions instead of points on a plane, will it behave the same way?

7 Robot actions

The same points, but they're Ada's actions

All lesson long the generator put points on spirals. For a robot a point is an action, and the question is the one the lesson started with: will the generator pick one of the answers it was shown, or their average?

Let's go back to the kettle detour from lesson 1.3. An action there is two numbers: Ada's sideways shift at step 1 and at step 2 of the detour, in centimeters. On the action plot this is a single point: horizontally the shift at step 1, vertically at step 2. In 400 demonstrations the operator went around the kettle sometimes on the left and sometimes on the right, so the points lie in two clusters: "both steps left" and "both steps right". Between the clusters is a band: actions where the total shift over the two steps is less than 30 cm (Ada's radius 18 cm plus the kettle's radius 13 cm), and Ada clips the kettle. The regression in lesson 1.3 output exactly such a point.

The denoiser from the lab doesn't care whether it's looking at spirals or actions: two numbers and a step number at the input, the predicted noise at the output. Train it on 400 actions instead of 2000 spiral points and generate 100 actions from noise. Alongside is the "noise → action" regression from the lab, trained on the same demonstrations: it gets the same 100 starting noises.

On the left is a diagram, top-down view: Ada on a wheeled base drives from bottom to top, a kettle in her path, two detours, left and right; circles 1 and 2 are the steps at which Ada shifts sideways. On the right is not a top-down view but an action plot: horizontally the shift at step 1 in cm (further left is left), vertically at step 2 (higher is right). Gray points are the 400 actions from the demonstrations, dashed circles are the clusters, the red hatched band is the actions where Ada clips the kettle.

Generation steps100

Under the hood: actions instead of spirals

Data. 400 actions: at a 50/50 share, 200 in each cluster. The cluster centers are a ±30 cm shift at both steps; the spread is Gaussian noise with standard deviation 5 cm. In network units 1 = 25 cm: centers ±1.2, spread 0.2, the same scale as the spirals.
Network and training. The same network as in the lab: two hidden layers of 64 neurons, Fourier features, step-number embedding. Adam, batch 125, 3000 steps; learning rate $3 \cdot 10^{-3}$ , after 1500 steps $10^{-3}$ , after 2250 $3 \cdot 10^{-4}$ . The difference from the lab: the step number $t$ during training is uniform, as in DDPM. The clusters have no fine structure to learn at low noise, and the action picks its side at high noise, where the network's accuracy matters more: when training was skewed toward small $t$ , as in the lab, a few of the 100 actions stayed between the clusters.
Generation. DDPM, 100 steps, 100 actions from the same starting noise. In free mode every generation starts from new noise.
Regression. The same network without the step number, 400 Adam steps on mean squared error: random noise at the input, a random action from the demonstrations at the output. At generation it gets the same 100 starting noises as the denoiser.
Criteria. An action is in a cluster if it's closer than 20 cm (four standard deviations) to its center, the dashed circle. An action is in the kettle band if the absolute sum of the two shifts is less than 30 cm. A weaving action, whose steps go in opposite directions, also falls into the band.

So the same network that put points on spirals puts actions into the clusters: of 100 actions, 50 went left and 50 right, none stayed in the kettle band, while regression on the same starting noises output a mean a centimeter from the center of the band, that is, a drive into the kettle. In lesson 1.5 this same model will drive Ada entirely: instead of two steps, an action chunk, and instead of the spiral number the condition will be the camera observation. What other misconceptions about diffusion are most common?

8 Check yourself

Eight common misconceptions

The questions are about the diffusion approach as a whole: what it solves, how training and generation work, and how this carries over to robot actions. Pick an answer and read the explanation.

9 Takeaways

Lesson takeaways

The forward process is fixed: noise is added by a schedule. By the end of the linear schedule $\bar\alpha_T \approx 4 \cdot 10^{-5}$ of the signal remains, and over the last third of the path the points are almost indistinguishable from noise.
The network learns to predict the added noise. The loss plateaus above zero: part of the noise can't be guessed in principle.
The denoiser points toward the data: at high noise toward the mean, at low noise toward the nearest spiral. So a sample ends up in one of the modes, not their average.
DDIM is the same network with deterministic steps. That's why robots take 4–16 inference steps, not 1000.
Classifier-free guidance is one network, with the condition sometimes dropped during training. Strong guidance trades diversity for precision.
The ideal denoiser only copies. On small data and with long training the network starts copying the training examples.

One problem remains. We generated individual points on a plane, and the condition was the spiral number. A robot needs something else: an action chunk several steps ahead, consistent in time, with a camera image as the condition. If every action is generated separately, neighboring actions may pick different detours. How to turn diffusion into a policy that picks one detour and sticks to it is the subject of lesson 1.5.

Open questions

Why can a diffusion policy trained on a dozen demonstrations behave like a lookup table, and why is that bad for a robot?

On small data the denoiser is close to the ideal one: the estimate of the clean action is a weighted average of the training examples, and generation ends up on one of the recorded trajectories. In preprint 2505.05787, Diffusion Policy on small datasets output a trajectory from training even for a photo of a cat, and a simple lookup table did just as well at 0.0034 of the inference time. For a robot this means it won't invent new motions: in an unfamiliar scene it repeats the nearest demonstration, even if it doesn't fit there.

A robot has to react within 20 ms. Which ways to cut the number of steps do you know, and what does each one cost?

DDIM and solvers like DPM-Solver take the same network with fewer steps, but at very few steps they lose quality, as in the lab. Distillation and consistency models bring inference down to 1–3 steps (Consistency Policy: 21 ms versus 192 ms for DDIM at 15 steps), but need a separate, expensive training stage. Flow matching with straight paths gives 4–10 steps (lesson 1.8). You can also avoid waiting for inference: execute an action chunk and compute the next one asynchronously, as in Real-Time Chunking. Single-step heads are the fastest but lose noticeably in success rate: in one author's tech report a drifting head for GR00T N1 (NVIDIA's model for humanoids, 2025; lesson 3.7) sped the head up from 45 to 5 ms at the cost of a drop in success rate.

Paper guide

J. Ho, A. Jain, P. Abbeel, "Denoising Diffusion Probabilistic Models", NeurIPS 2020.

Read closelyHow the forward process works and why $x_t$ is obtained from $x_0$ in one step; the noise parameterization and the $L_{\text{simple}}$ objective (equation 14); the parallels with denoising score matching and Langevin dynamics.
Safe to skipThe detailed derivations of the variational bound and the network architecture details for images.
Question to keep in mindHow many network calls does it take to generate one image, and why did the authors pick $T = 1000$ ?

Resources

Yandex Machine Learning Handbook, chapter 8.5Diffusion models, in Russian
L. Weng, What are Diffusion Models?An overview with derivations in DDPM notation
Y. Song, Generative Modeling by Estimating Gradients of the Data DistributionThe score-based view
S. Dieleman, Guidance: a cheat code for diffusion modelsClassifier-free guidance in detail
Welch Labs for 3Blue1Brown, 2025A video on how diffusion models work
MIT 6.S184MIT's course on diffusion and flow matching; lecture notes at arXiv 2506.02070
tiny-diffusionA small implementation of diffusion on 2D data
Diffusion literature for roboticsA list of papers on diffusion in robotics

➡️

Next: Diffusion Policy. We'll replace the points with an action chunk and make the camera observation the condition. Then the initial noise will decide whether the robot goes around the kettle on the left or on the right, and denoising will carry the detour through to the end — lesson 1.5, "Diffusion Policy".

Was the lesson clear?

Found a mistake or inaccuracy? Report it on GitHub

Found this lesson useful? Say thanks — that is how the author knows the course is being read.