1.5 Part 1 · Imitation learning

Diffusion Policy

In lesson 1.3 we found that a policy needs a model of the action distribution; otherwise it averages the detour on the left and the detour on the right and drives straight into the kettle. In lesson 1.4 diffusion learned to draw samples from a distribution of any shape. Diffusion Policy combines the two ideas: diffusion builds not a single point but a chunk of future actions, and the latest observations serve as its condition. The initial noise picks a strategy, and the chunk holds it. In this lesson we'll train such a policy for Ada, find out how many steps of a chunk to execute, how many denoising steps are really needed and why the chunk consists of positions rather than velocities, and look at why the method works.

Ada is the home humanoid robot the course labs are built around. More about her in lesson 0.1.

≈ 50 minutesa lab and 4 interactives11 missions with a goal

PUSH-TPut the T-block into the goal and see how the benchmark computes the reward

DENOISINGScrub through the denoising of a chunk and find the step at which the detour side is chosen

LOOPAssemble the policy step by step, from the camera to the motor, and discard the extra blocks

LABTrain the policy in your browser and test chunks, denoising steps and the execution horizon

Start the lesson ↓

1 Task

Push-T: the task from the paper

Picking one strategy isn't enough for a policy: it has to execute it smoothly, on a real robot and in real time. In 2023 Chi et al. proposed Diffusion Policy for this and tested it on 12 tasks from four benchmarks: Robomimic, Push-T, Block Pushing and Franka Kitchen. The paper's main example is Push-T.

In Push-T a round agent, a pusher controlled by the policy or by a person with a mouse, pushes a T-shaped block into a green outline on the table. The task is multimodal: you can approach the block from the left or from the right and push it by the crossbar or by the stem, and operators do it differently. On a UR5 industrial arm Diffusion Policy solved the task in 95% of trials. Earlier methods did worse: IBC (an implicit policy based on an energy model) never succeeded, and LSTM-GMM (a recurrent network with a mixture of Gaussians at the output) succeeded in 20%.

Before comparing policies, let's sort out the metric: what exactly counts as a solution. The reward is the fraction of the goal covered, with a margin: once the block covers 95% of the goal, the reward is already 1. There's no agent in the interactive; you move the block yourself.

0%goal coverage · line at 95%100%

Top view of the table: you move the gray T-block, and the green dashed outline is the goal. The bar under the scene shows what fraction of the goal is covered.

Rotation−20°

The metric says where to end up, but not how to get there. How does a policy pick one way out of several and not abandon it halfway?

Under the hood: the Push-T reward

The reward is the fraction of the goal covered, with a 5% margin:

r = \min\Bigl(1,\ \frac{S_{\text{covered}}}{0.95\, S_{\text{goal}}}\Bigr).

The benchmark metric is the best reward per episode, averaged over 50 starts. On the real robot the authors measured the overlap of the block with the goal at the last step (IoU): 0.80 for Diffusion Policy, 0.84 for a human operator.

In the interactive the block and the goal consist of a 140 × 35 crossbar and a 35 × 100 stem, and the goal is rotated by 45°. The intersection area is computed exactly: the block and the goal are two rectangles each, and the intersection of convex polygons is found by Sutherland–Hodgman clipping.

2 Chunk

Diffusion builds an action chunk

In lesson 1.4 diffusion generated a point on a plane. A policy needs not a point but a motion. So Diffusion Policy generates an action chunk: the robot's positions 16 steps ahead. The whole chunk starts as Gaussian noise and turns into a coherent motion in a few denoising steps.

The observations are not noised. Camera frames and the robot's position are only the condition: the network receives them at every denoising step and uses them to decide where to move the chunk. The generation procedure is the same in any situation; only the condition changes.

Why does a chunk hold a strategy? The detour side is chosen once for the whole chunk, and all 16 steps follow that choice. Scrub through the denoising of a chunk next to the kettle and find the step at which the side is chosen. We count denoising steps from one: step 1 is the first estimate of the clean chunk, which the network makes from pure noise, and step 16 is the finished chunk. The side counts as chosen from the step at which the clean-chunk estimate at the kettle's level has left the corridor extending 6 cm on either side of the route and stays on one side until the end of denoising: a smaller deviation is still noise, not a choice.

Top view: Ada at the bottom drives upward and has to go around the kettle. The orange dots are the chunk: Ada's 16 future positions, which denoising gradually cleans of noise. The dashed line is the estimate of the clean chunk at this step, and the ring on it is the estimate's point at the kettle's level: the side is judged by that point. The gray band along the route is the ±6 cm corridor: while the ring is inside it, no side is chosen.

Cells 1–16 are denoising steps (step 1 is the first estimate from pure noise); the color is the side of the clean-chunk estimate at that step:detour on the leftdetour on the rightno side chosen

Denoising step0

So the strategy is picked by the initial noise, and denoising carries the chunk through to one of the strategies present in the data. How do you turn this into a policy that works on a robot, from the camera to the motor?

Under the hood: denoising a chunk with a condition

Let's denote the chunk at denoising step $k$ by $a^k$ and the observations by $o$ . The network $\varepsilon_\theta$ is trained to guess the added noise, as in lesson 1.4, only with the observations as an input:

\begin{gathered} L = \E\,\bigl\lVert \varepsilon - \varepsilon_\theta(a^k, k, o)\bigr\rVert^2,\\ a^k = \sqrt{\bar\alpha_k}\,a^0 + \sqrt{1 - \bar\alpha_k}\,\varepsilon. \end{gathered}

A DDIM step from $k$ to an earlier $k'$ first estimates the clean chunk, then adds the predicted noise back to it at a lower level:

\begin{aligned} \hat a^0 &= \frac{a^k - \sqrt{1 - \bar\alpha_k}\,\varepsilon_\theta(a^k, k, o)}{\sqrt{\bar\alpha_k}},\\ a^{k'} &= \sqrt{\bar\alpha_{k'}}\,\hat a^0 + \sqrt{1 - \bar\alpha_{k'}}\,\varepsilon_\theta(a^k, k, o). \end{aligned}

In the interactive the denoiser is exact and has no condition, as in lesson 1.3: the chunk is 16 sideways offsets of Ada around the kettle, 9 steps before it and 6 after, built from 40 demonstrations. We judge the side by the clean-chunk estimate $\hat a^0$ at the kettle's level: more than 6 cm to the right of the route means right, more than 6 cm to the left means left, otherwise no side is chosen. The choice step is the first step from which this side stays the same until the end. The lab below uses the same 6 cm dead zone to count plan switches. In the lab the denoiser is a trained network with a condition.

3 Loop

How the policy is built

One chunk for the whole task isn't enough: the scene changes, and the robot has to react. So the policy runs in a loop. It looks at the two latest frames, the encoder computes their features, denoising builds a chunk of 16 actions, the robot executes the first 8, and the loop repeats with new frames. This is the receding horizon you already met in lesson 0.3: plan far ahead, execute the beginning, re-plan.

Assemble this loop yourself. Two of the blocks are extra.

Policy loop

Blocks

The authors chose the numbers 2, 16 and 8 empirically. A short observation history overfits less, a long chunk holds the strategy, and executing half of the chunk leaves time to react. Let's check how this works on Ada.

Under the hood: the policy from the Diffusion Policy paper (Chi et al., 2023)

This is how the policy from the Diffusion Policy paper (Chi et al., 2023) is built in its UNet variant, which the authors recommend as the starting point; the numbers are from the paper.

Input. The two latest observation steps: frames from two cameras and the gripper position.
Encoder. A ResNet-18 per camera without pretraining, spatial softmax instead of global average pooling, GroupNorm instead of BatchNorm. The features are computed once per chunk inference.
Denoiser. A 1D UNet along the time axis of the chunk; the observations and the step number enter through FiLM. There is also a transformer variant with cross-attention to the observations, but the authors recommend starting with the UNet.
Training. A random step out of 100, the squared error between the noise and its prediction, actions normalized to the interval [−1, 1], an exponential moving average of the weights (EMA).
Control. The chunk holds target positions, not velocities. For the authors position control was consistently better than velocity control; for the baselines it was the other way round.
Size and rate. The noise network has 256M parameters in simulation and 67M on the robot, the encoder 22M for two cameras. The policy outputs a chunk 10 times per second, and the controller interpolates it to 125 Hz.

🔬 Lab

Ada and the diffusion policy

Let's check it on the scene from lesson 1.3. The policy sees Ada's two latest positions and outputs a chunk 16 steps ahead. The denoiser is a real small network that trains right in your browser in a few seconds. Next to it you can turn on regression with chunks: it outputs the whole chunk in one network pass.

Horizontal axis: how many of the chunk's 16 steps to execute. Vertical axis: how many of 20 runs passed with no touch and no plan switches when the kettle shifts.

show the chunk add regression shift the kettle

Share of left detours50%

Denoising steps16

The chunk holds the chosen side, executing half of the chunk is a compromise between commitment and reaction, and far fewer denoising steps are needed than in training. In the lab the chunk consisted of Ada's positions, not velocities. It's the same trajectory written differently, and yet the authors chose positions. Why?

Under the hood: the policies in the lab

Observation. Ada's positions at the previous and the current step relative to the kettle: four numbers. Ada estimates her sideways position with an error of about 8 cm, as if from an imprecise sensor.

Chunk. Ada's positions 16 steps ahead relative to the current one, in $x$ and $y$ : 32 numbers. We execute $T_a$ steps out of 16, then build a new chunk.

Denoiser. A fully connected network: the input is the noisy chunk, the observation and the noise step number (43 numbers), then two layers of 64 neurons with tanh, and the output is the estimate of the clean chunk. A cosine schedule, 100 noise steps in training, 3000 Adam steps with batches of 32: about two seconds in the browser. At inference, DDIM with the chosen number of steps.

Regression with chunks. A 4 → 64 → 64 → 32 network, mean squared error, 2500 Adam steps. The whole chunk in one pass.

Runs. Each launch is 10 runs: the start is slightly shifted, and controller noise of about 0.5 cm is added to the motion. In the mission with the kettle shift, each launch is the same 20 runs for any execution horizon: the kettle shifts 12 cm toward Ada's detour side at a step between the 8th and the 19th. That way execution horizons are compared on equal terms.

Metrics. A touch: the centers of Ada and the kettle come closer than 31 cm. A chunk's side is the mean sideways offset of its points from the kettle while the kettle is no farther than 60 cm from them: more than 6 cm to the right means right, more than 6 cm to the left means left, otherwise no side is chosen. A plan switch: a new chunk with a side goes around the kettle on a different side than the previous chunk with a side; chunks without a side don't count.

🧩 Interactive

Position or velocity: what the policy outputs

A chunk of 16 positions says where to be at each step. The same chunk can be written as 16 velocities: how far to move at each step. The trajectory is the same, but the robot's controller executes the two encodings differently, and Chi et al. chose positions for a reason.

The controller is a program on the robot that turns the policy's command into motor currents and runs much more often than the policy: in the paper the policy outputs a chunk 10 times per second, and the controller executes it at 125 Hz. A velocity command says “move by this much”, and the controller counts the shift from where the arm actually ended up: if the previous step missed by a centimeter, that centimeter stays, and the misses add up. A position command says “go to this point”: the controller brings the arm there on its own, and the previous miss is erased by the very next step.

Latency adds a second difference. The chunk is computed from frames taken several steps ago, and by the time it arrives, the robot is no longer where it was when the chunk was computed. A velocity command is executed from the new place and leads off course, while the correction to the route, which sits in the chunk's first commands, arrives when their steps have already passed. A position is an absolute target: a late or skipped command only means that the controller heads for the same point a little later. Hence the link to chunks: a chunk of positions sets a trajectory the controller returns to on its own, while a chunk of velocities only sets the shape of the motion from wherever the robot ends up. In the paper position control is consistently better than velocity control for Diffusion Policy, while for earlier methods it was the other way round. The authors attribute this to the multimodality of the demonstrations being more pronounced in positions: it hampers methods without a model of the distribution, but not diffusion.

Check this on Ada. The policy isn't trained here: the chunk of 16 positions is built by the teacher that recorded the demonstrations in lesson 1.3, and it steers Ada back onto the detour route. The same chunk is executed by two controllers under latency and slight execution noise. The lab engine doesn't support this mode, so the execution model here is separate and simplified; it's described in “Under the hood” below.

Top view of Ada's home, as in the lab: the start at the bottom, the table at the top, the kettle in the way. Two Adas start together and get the same chunks with the same execution misses: the orange one executes the chunk as positions, the blue one as velocities. The dashed line is the intended path, that is, the chunks' points step by step; the dots ahead of Ada are her current chunk: filled ones are executed, outlined ones are ahead, gray ones were skipped because of the latency. Thin lines are the other nine runs.

Chunk latency3 steps Execution noise1 cm

A chunk of positions sets a trajectory the controller returns to on its own, so latency and execution misses barely hurt it. But the latency itself hasn't gone away: while the network computes, the robot executes the old chunk. Where does the latency come from, and how many denoising steps do real robots take?

Under the hood: the execution model in the interactive

Chunk. From Ada's observed position, the teacher from lesson 1.3 builds 16 positions ahead: at each step the shift is $0.65\,(x_{\text{route}} - x)$ plus the route's own shift, at most 9 cm per step. The observation is exact. 8 steps out of 16 are executed, and then a new chunk is needed.

Latency. A new chunk is requested when execution of the current one starts and arrives $d$ steps later; until it arrives, Ada continues the old chunk by step number. The new chunk's commands for steps that have already passed are skipped; on the scene these are the gray dots.

Controllers. Position: $x \leftarrow x + (p_j - x) + \varepsilon$ . Velocity: $x \leftarrow x + (p_j - p_{j-1}) + \varepsilon$ , where $p_j$ are the chunk's points and $\varepsilon$ is a Gaussian miss with the given spread in centimeters. The sequence of misses is the same for both controllers and for any settings: ten runs with seeds 1–10. Ada moves forward 6 cm per step without noise.

Deviation. Where Ada ended up after a step minus the chunk point she was executing at that step. “At the kettle” is the step at which Ada is closest to the kettle along the way, “at the end of the path” is the last step; both are averaged over the ten runs. A touch is the same as in the lab: the centers come closer than 31 cm.

The model is simplified: there's no inertia and no acceleration limit, the observation is noise-free, and the latency is the same for all chunks. The paper compared control of a real UR5 arm, where the actuator dynamics add to these effects.

4 Speed

How many steps real robots take

Each denoising step is a full pass of a large network. DDPM with 100 steps spends about a second on a T4 GPU, while a robot at 10 Hz waits for a new chunk every 0.6–0.8 seconds. In the lab Ada got by with three or four steps, and real systems go the same way: they cut the number of steps, change the generator and build the next chunk while the current one is executing.

The table is a reference for later: most of the systems in it are covered in the next lessons, and the lesson number is given in parentheses.

System	Generator	Inference steps	Inference time
Diffusion Policy, 2023 — the policy of this lesson	DDIM	16 (10 in section 3.4 of the paper)	0.1 s on an RTX 3080
Consistency Policy, 2024 — a distilled Diffusion Policy	distillation	1–3	21–22 ms vs ~195 ms for DDIM with 15 steps
RDT-1B, 2025 — a bimanual diffusion policy with 1.2B parameters	DPM-Solver++	5	—
π0, 2024 — Physical Intelligence model (lesson 3.6)	flow matching	10	73 ms on an RTX 4090
GR00T N1, 2025 — NVIDIA model for humanoids (lesson 3.7)	flow matching	4	63.9 ms on an L40
ALOHA Unleashed, 2024 — Google DeepMind diffusion policy on a bimanual robot (lesson 1.7)	DDIM	50	43 ms on an RTX 4090

The table is wider than the screen: you can scroll it sideways.

Flow matching with Gaussian noise is equivalent to diffusion but learns straighter paths from noise to data, so it needs fewer steps: π0 (Physical Intelligence policy, 2024; lesson 3.6) takes 10 steps, GR00T N1 (NVIDIA model for humanoids, 2025; lesson 3.7) takes 4. We'll cover flow matching in lesson 1.8. Fast inference still doesn't remove the latency entirely: while the network computes, the robot has to execute something. What happens at the seam between chunks?

5 Seams

Chunk seams and latency

While the network builds the next chunk, the robot either waits or continues the old one. Synchronous inference stops the robot, and the motion comes out with pauses. Asynchronous inference continues the old chunk and switches to the new one when it's ready. But the new chunk was built from outdated frames and may pick a different strategy, so a jump appears at the seam.

You can't smooth the jump by averaging, for the same reason as in lesson 1.3: the average of two strategies can be an invalid action, and the π0 authors rejected averaging. Real-Time Chunking (RTC) from Physical Intelligence works differently. The next chunk is built while the current one is executing; the actions that are already going to the motors are frozen, and the rest are inpainted so that the new chunk continues the one being executed. The method works without retraining and withstands latencies above 300 ms. The diagram below shows both approaches side by side: RTC on the left, naive chunk replacement on the right.

Both panels show the same picture: horizontally, control steps, that is, time; vertically, Ada's sideways offset from the straight line as she goes around the kettle. The orange dots are chunk 1 of 16 actions, and they go to the motors one at a time: filled ones are already executed, the black line is the executed path, and the “motors” strip at the bottom shows which actions have already gone out. At step 4 the network starts building chunk 2 and is busy for 4 steps (the bar at the top), while the robot keeps executing chunk 1. On the left, Real-Time Chunking: the first 4 actions of chunk 2 are frozen (gray squares with a lock), copies of the actions executed while the network was computing; the rest, in purple, are inpainted so as to continue the executed line, and there's no jump at the seam. On the right, naive replacement: chunk 2 is built from new noise using the frames of step 4, picks a different detour, and at step 8 the path jumps (red cross). Click the diagram to pause it.

We'll cover latency and ways of stitching chunks in detail in lesson 1.7, using ACT (a policy for the bimanual ALOHA robot, 2023) as the example. What remains is to understand why a diffusion policy works better than others at all, and whether the authors' explanation holds.

6 Why it works

Why the method works

The authors give three explanations. The first is multimodality: the initial noise picks a strategy, denoising carries it through to the end, and there's no averaging. The second is the long output: diffusion describes a distribution over the whole chunk, so it can predict a consistent sequence of actions. The third is stable training: IBC has to estimate the normalization, a sum over all possible actions, from negative examples, while diffusion learns the score (the direction in which to move a point, from lesson 1.4) and doesn't need the normalization.

Results, however, have to be kept apart from explanations. The average gain of 46.9% is a comparison in simulation. On real robots the authors report individual tasks: 95% on Push-T, 90% on the mug task. The paper doesn't prove that the gain comes from multimodality specifically.

This explanation is disputed. According to Mazza et al., standard benchmarks have almost no conditional modes (lesson 1.3). A 2025 paper (arXiv 2512.01809) on 28 benchmarks attributes the advantage of generative policies to noise during training and iterative steps rather than to the model's expressiveness: a two-step regression policy was barely worse than a flow policy. And when demonstrations from different operators really do diverge, a generative head is needed: in ALOHA Unleashed (Google DeepMind bimanual tasks, 2024; lesson 1.7) diffusion reached 70% success versus 25% for regression with L1.

The practical takeaway is this. If the data has one strategy, regression with chunks is cheaper and no worse. If operators solve the task in different ways or the scene allows several solutions, you need a generative policy.

7 Check yourself

Eight common misconceptions

8 Takeaways

Lesson takeaways

Diffusion Policy generates an action chunk 16 steps ahead from noise, conditioned on the latest observations. The observations are not noised.
The strategy is picked by the initial noise in the first denoising steps, and the chunk holds it, so the robot doesn't weave.
Receding horizon: predict 16, execute 8. Execute fewer, and the plan switches sides; execute more, and the robot notices scene changes late.
The chunk is positions, not velocities: the controller drives toward an absolute target, execution misses don't accumulate, and a late command does less harm. In the interactive, with a 3-step latency, velocities drifted from the chunk several times more than positions.
Few denoising steps are needed: 16 for Diffusion Policy, 10 for π0, 4 for GR00T N1, 1–3 after distillation. One step without distillation gives almost the mean.
Chunk seams are not averaged: the next chunk is built onto the one being executed, as in Real-Time Chunking.
Multimodality is the authors' explanation, not a proven cause of the success. If the data has one strategy, regression with chunks is cheaper and no worse.

Open questions

Inference takes 100 ms, and the robot is controlled at 10 Hz. How do you organize execution so that there are neither pauses nor jerks?

Build the next chunk asynchronously while the current one is executing, and start inference early, as many steps before the end of the chunk as inference takes. Freeze the actions that will be executed during inference and inpaint the rest of the chunk onto them, as in Real-Time Chunking, instead of averaging chunks. The latency can also be simulated during training. In Hugging Face's LeRobot library, asynchronous inference cut the time per task from 13.75 to 9.7 s.

The policy outputs positions, and the controller drives toward them on its own. What happens at contact, when Ada needs not to reach a point but to press on an object?

Through a stiff controller, a target inside an object turns into a force the policy never specified: the controller keeps pushing until it reaches the point. In practice the position is executed by a compliant controller with a force limit, and for contact-rich tasks policies sometimes output not only a position but also a stiffness or a force. How such controllers live inside the loop of a trained policy is beyond the scope of this lesson.

Paper guide

C. Chi et al., “Diffusion Policy: Visuomotor Policy Learning via Action Diffusion”, RSS 2023.

Read closelyThe receding horizon and the choice of the observation, prediction and execution lengths; the comparison of position and velocity control; the experiments on the real Push-T.
Safe to skipThe details of the transformer variant and the tables for individual Robomimic tasks.
Question to keep in mindWhy do the authors recommend starting with the UNet variant rather than the transformer?

Resources

Diffusion PolicyProject site: receding-horizon videos on Push-T and the mug task
Diffusion Policy codeThe authors' implementation, MIT license
The authors' ColabTraining a policy on Push-T in your browser
Cheng Chi's talkThe first author on the method and the experiments
Real-Time ChunkingK. Black et al., 2025 — how to stitch chunks without pauses or jerks
Atlas and Large Behavior ModelBoston Dynamics and TRI, 2025 — a diffusion policy on a humanoid

➡️

Next: the Transformer. The UNet variant of Diffusion Policy works well with one or two cameras. Modern policies take many cameras, a text command and an observation history as input, and almost all of them are built on the transformer. How it works and why it suits policies — lesson 1.6, “The Transformer”.

Was the lesson clear?

Found a mistake or inaccuracy? Report it on GitHub

Found this lesson useful? Say thanks — that is how the author knows the course is being read.