1.3 Part 1 · Imitation learning

Action multimodality

In lesson 1.1 Ada learned to drive around the apartment by repeating after the teacher, and she went astray where there were no examples. In lesson 1.2 DAgger added examples exactly in the places she drifts to. Both times data helped. In this lesson a kettle stands in Ada's way, and the teacher goes around it sometimes on the left, sometimes on the right. There are plenty of examples, all of them correct, yet after training Ada drives straight into the kettle, and new drives by the teacher don't help. Let's figure out why two correct examples give a wrong answer, which models can pick one of several correct actions, and how to avoid changing that choice halfway.

≈ 50 minutes3 labs, 4 interactives and a quiz15 missions with a goal

EXPERIMENTTrain Ada on 40 teacher drives past the kettle, bet on the outcome, and check whether 1000 drives help

CAUSEDraw the detours yourself and see which path a network learns from them

DATAEstimate which datasets really contain several strategies, and get a score for your confidence

THREE ADASThree policies drive to the same kettle at once: you'll see which of them picks a side and which one averages

CHUNKSPick how many steps ahead Ada should plan, and hold the chosen side where one plan gives way to the next

Start the lesson ↓

1 Experiment

More demonstrations, fewer crashes?

Ada drives from the living room to the kitchen, to the table, and a kettle stands on the straight path. The drives are recorded by the teacher, a program that knows the way. It goes around the kettle sometimes on the left, sometimes on the right, and a coin picks the side. As in lesson 1.1, the network learns to repeat after the teacher: given what fraction of the path Ada has already covered, it outputs how far to shift sideways on the next step.

Ada is a home humanoid on a wheeled base; the course labs are built around her. More about her in lesson 0.1.

Train the network and see how Ada drives. Then add data: in lesson 1.1 new drives filled the gaps, and the errors went away. Here there are no gaps: every one of the teacher's drives goes past the kettle. Place your bet before you run it.

🔬 Lab 1

A top view of Ada's home. Ada (the circle with eyes) starts at the bottom, in the living room, and drives up to the table in the kitchen; the dashed line is the straight route, the kettle stands on it, and the dashed circle by the table is the goal. Gray lines are the teacher's drives; after a run, green lines are Ada's drives without a touch, and red ones with a cross are drives that touch the kettle.

There's more data, but no fewer touches: the number of examples isn't the issue. The network saw only detours, yet it chose a path that wasn't in any of the teacher's drives. Why?

Under the hood: how the experiment works

The scene. Ada is a circle with a radius of 18 cm, the kettle 13 cm. A touch counts if the centers come closer than 31 cm. Ada drives up 6 cm per step of 0.1 s; without the kettle the route is a straight line from the start to the table, 52 steps.

The teacher. It first drives straight, then turns and passes the kettle on average 56 cm from the route, with a spread from 52 to 62 cm, and returns to the route before the kitchen doorway. At every step the teacher's path jitters a little, by about 0.5 cm.

The network. The observation is one number: what fraction of the path Ada has already covered. The output is the sideways offset from the route at the next step. A 1 → 32 → 32 → 1 network with tanh, the MSE loss function, 400 Adam steps with batches of 32 examples. A controller steers Ada toward the predicted offset at no more than 9 cm per step. In each of the 20 trials the teacher records new drives with a new seed, the network learns from scratch, and Ada drives once.

Why the network sees only the path fraction. That way all demonstrations at one point of the route have the same observation but two actions, and the cause of the crash shows up in pure form. If you also give the network Ada's sideways position, the closed loop often picks a side on its own: the network amplifies a small deviation from the middle, because next to it all demonstrations go the same way. This happens in practice too: in the CARLA driving simulator, with 50/50 demonstrations, regression turned right in 98% of runs. Regression can pick a side, but nothing guarantees it.

Our own math. The built-in Math.exp, Math.log and Math.tanh differ in the last bit across browsers, and training inflates that difference into different numbers. On this page they are replaced with our own versions built from additions and multiplications, so the numbers in the labs are the same in any browser.

2 Cause

Regression outputs the mean

The answer lies in what the loss function teaches. That's the name for the number the network reduces during training: how far its answers are from the examples. A network trained with mean squared error (MSE) looks for the action that is, on average, closest to all the examples with the same observation. This is the conditional mean. As long as there is one correct action, the mean coincides with it. If there are two correct actions, a detour on the left and a detour on the right, the mean ends up between them, that is, in the kettle. That's exactly what happened in the experiment: the more teacher drives, the closer the split between the sides gets to 50/50 and the more precisely the network hits the middle.

Christopher Bishop described this in 1994: “the average of several correct target values is not necessarily itself a correct value”. His example is the inverse kinematics of a two-link arm. It's the same error as in lesson 0.3: one gripper pose has two solutions, with the elbow one way or the other, and the network output their average, which missed the target.

The first thing that comes to mind is to change the loss function. With L1, the sum of absolute deviations, the network outputs the median instead of the mean. OpenVLA-OFT (an open policy, Stanford, 2025; lesson 3.5) is built on this: regression with L1 reaches 97.1% success on the LIBERO benchmark. But its authors point out that in their data each task has a single strategy.

Draw the detours yourself and see what MSE and L1 do. Draw a line from Ada to the table past the kettle: each line is one demonstration. The solid line is the answer of a network with MSE, the dashed one with L1.

🧩 Interactive

Top view: Ada at the bottom, the table at the top, the kettle on the straight route; the dashed circle around the kettle is where Ada's center must not go. Gray lines are the demonstrations you drew; the dark solid line is the answer of the network with MSE (the mean of the lines), the dashed one is the answer with L1 (the median). The small circle by the kettle labeled MSE or L1 is where Ada draws level with the kettle: green means she gets past, red means she clips it.

So L1 picks the majority side, but with an even split its answer is undefined. MSE leads into the kettle at any split except a strong skew. We need a model that keeps both sides and picks one of them. But how often does real data have several correct actions for one observation?

Under the hood: mean and median

For MSE the optimum is the conditional mean, for L1 the conditional median:

\begin{aligned} \min_f \E\,\bigl\lVert a - f(s)\bigr\rVert^2 &\;\Rightarrow\; f(s) = \E[a \mid s],\\ \min_f \E\,\bigl\lvert a - f(s)\bigr\rvert &\;\Rightarrow\; f(s) = \operatorname{med}(a \mid s). \end{aligned}

The minimum of the sum of squares is the mean: shifting the answer in any direction only increases the sum. The minimum of the sum of absolute values is the median. Shifting the answer by a small $\delta$ changes the sum by $(n_1 - n_2)\,\delta$ , where $n_1$ and $n_2$ are the numbers of points on either side of the answer, so it pays to move toward the majority. With an even split the sum stays the same along the whole segment between the two middle points, and the solution isn't unique. Here the L1 dashed line is drawn in the middle of that segment, and the segment itself is shaded.

No network is trained here: we draw its optimum right away. For MSE it's the mean offset of all lines at each point of the path, for L1 the median. A trained network arrives at the same lines: in the experiment above it drove Ada exactly along the mean of the teacher's drives. Each line you draw is smoothed and, if you release it after the kettle, extended to the table.

3 Multimodality in data

Where data contains several strategies

Each separate correct way to act is called a mode of the action distribution, and a situation where one observation has several correct actions is called action multimodality. The kettle has two modes, a detour on the left and a detour on the right, because the teacher flips a coin. In real data several correct actions appear for reasons of their own. The diagram shows five of them, each as a mini-scene with a top view of a table.

Five sources of several strategies, top view of a table. The bracket with two jaws is the robot's gripper, controlled by an operator; the small circle with a handle is a cup, the target. Different operators: A goes around the bowl on the left, B on the right; in the robomimic dataset six operators recorded 50 demonstrations each, and on such data a mixture of Gaussians works noticeably better than a single action: with a deterministic output, results drop by up to 58% in relative terms. Subtask order: in Franka Kitchen the operator completes 4 of 7 subtasks in varying order. Pauses: the operator freezes, so at the same spot the data sometimes has motion and sometimes a stop; the Diffusion Policy authors (Chi et al., 2023; lesson 1.5) call pauses “common during teleoperation”. Spread within one operator: in ACT (a policy for the two-armed ALOHA robot, 2023; lesson 1.7) the spot where one arm hands an object to the other is different every time. The operator sees beyond the frame: outside the frame there's a hot pot, and the operator drives around it, while the frame looks the same as on the straight path; to the policy such decisions look random. Click the diagram to pause it.

Hence a popular argument: generative policies, that is, policies that output a distribution of actions and pick one action from it, win because the data is multimodal. Mazza et al. tested it directly in 2026. They estimated the number of conditional modes $\hat K$ : how many different continuations one observation has on average over a horizon of 10 steps. $\hat K = 1$ means there is one strategy.

Before looking at the numbers, make your own estimate. For each dataset, set the probability that one observation in it has noticeably more than one strategy, that is, $\hat K$ of at least 1.1. The more confident you are in a correct answer, the more points you get; the more confident in a wrong one, the fewer.

🧩 Interactive

Standard benchmarks have almost no modes, and regression does no worse there than generative models: Pan et al. compared them on 28 benchmarks with the same architecture and found no difference, except on high-precision tasks. Data from different operators does have modes: in ALOHA Unleashed (two-armed tasks from Google DeepMind, 2024; lesson 1.7), on 8658 demonstrations diffusion reaches 70% success versus 25% for L1 regression. When there are modes, how do you teach a network to output a distribution of actions instead of a single number?

Under the hood:

\hat K

estimates and the score

Over a horizon of 30 steps the estimates are: Push-T 1.01, LIBERO 1.03, MetaWorld 1.07, Kitchen 1.23, a real bimodal grasp 2.27. The real grasp is a surgical robot taking tissue to the left or to the right of a danger zone: 90 demonstrations, 45/45. Deterministic regression gets 0% success on it.

The score is the Brier rule: for each dataset the penalty is $(p - y)^2$ , where $p$ is your probability, $y = 1$ if $\hat K \ge 1.1$ , and $y = 0$ otherwise. Points: $100 - 20 \sum (p - y)^2$ . Answering “50%” everywhere gives 75 points, full confidence in the correct answers gives 100, full confidence in the wrong ones gives 0.

We haven't found estimates of what fraction of states is multimodal in large datasets such as DROID and Open X-Embodiment (lesson 3.4).

4 Models

How to output a distribution of actions

To avoid averaging, a policy has to output a distribution of actions and draw a sample from it, that is, pick one action at random according to the probabilities. There are several ways to do this, and each has its cost. This lesson covers five: regression, a mixture of Gaussians, bins, autoregression and diffusion. Autoregression extends bins: the action dimensions are picked one at a time, and each next one depends on those already picked; it isn't on the diagram, it's shown in the interactive below. The diagram shows what each of the others outputs on the same data, and the table shows their ideas and costs.

The axis is the sideways shift when going around the kettle. Gray is the distribution of actions in the demonstrations: a detour sometimes on the left, sometimes on the right, two peaks. Orange is what the policy outputs; a dot on the axis is one chosen action. Regression outputs the mean of the two detours and hits the kettle. The other three methods pick one of the sides, anew each time. Click the diagram to pause it.

Method	Idea	Strength	Cost
Regression: MSE or L1	one action: the mean or the median	fast and stable	averages the modes or picks one at random
Mixture of Gaussians	several weighted Gaussian bells, each describing one mode: Bishop 1994, robomimic 2021	explicit modes, cheap sampling	the number of components $K$ is set in advance
Per-dimension bins	each action dimension is split into intervals, and a classifier is trained; RT-1 (Google, 2022; lesson 3.1) uses 256 bins for each of its 11 dimensions	any distribution shape along one axis	loses the links between dimensions
Tokens and autoregression	the action is written as text and output token by token: RT-2 (Google DeepMind, 2023; lesson 3.3), OpenVLA (an open model, 2024; lesson 3.5), FAST (a Physical Intelligence tokenizer, 2025; lesson 3.6)	a vocabulary shared with the language model	slow inference
Diffusion	random noise is turned step by step into a whole drive that resembles one of the demonstrations: Diffusion Policy (lessons 1.4 and 1.5)	stable training, modes of any shape	dozens of network passes per decision

The choice of method shows in practice. In RT-1, replacing the 256 bins with a single Gaussian drops success from 97 to 68% on familiar tasks and from 76 to 43% on new ones. Three more methods and their comparisons are in the block below.

Under the hood: three more methods and their cost

The same data as on the diagram above: gray shows the two detour peaks, orange what the policy outputs. Tokens one by one: an action or an action chunk is written as a sequence of tokens, which are output one after another, and each next token sees the previous ones, so all three steps land on the same side. Clusters and offset: the centers of action clusters (triangles) are found in advance; the policy picks a center and adds a small continuous offset. Energy: instead of an action, the policy outputs the energy of every action (the orange curve, lower is better); candidates roll into the nearest well, and the lowest one is executed. Latent style: in training the variable z encodes the style of a demonstration and moves the answer between the modes; at test time z = 0, and the answer is the same every time. Click the diagram to pause it.

Method	Idea	Strength	Cost
Clusters and offset	BeT 2022: k-means over actions plus a continuous correction	picks a mode in one pass	the number of clusters k is set in advance
Energy	IBC 2021: the action is the minimum of the energy E(s, a)	keeps discontinuities sharp	unstable training, expensive inference
Latent style	CVAE in ACT (lesson 1.7): the latent z encodes the operator's style, and the policy outputs an action chunk	fast, needs little data	at test time z = 0, that is, one mode

A method related to diffusion is flow matching: noise is also turned into an action plan, only along a straighter path; π0 (a Physical Intelligence policy, 2024; lesson 3.6) is built on it. We'll cover it in lesson 1.8.

In IBC on a real xArm6 arm, the energy model versus MSE: 85.0 versus 35.0% on two blocks, 88.3 versus 55.0% on the “any targets, any order” variant, 83.3 versus 6.7% on insertion with a 1 mm tolerance. ACT without the CVAE drops from 35.3 to 2% on human demonstrations, while on scripted ones there's no difference.

The simplest method is bins. Each action dimension is split into intervals, called bins, and a classifier is trained to give the probability of each interval. Such a table describes any distribution shape along one axis, even one with two peaks. But an action usually has several dimensions, and if each one is picked from its own table independently, the model doesn't know how they are related. Check this on an action made of two numbers: Ada's sideways shift on two consecutive steps of the detour. The second mode of the interactive is autoregression: the dimensions are picked one at a time, and each next one depends on those already picked, the way a language model picks words one after another.

🧩 Interactive

This is not a top view but a plot of actions. Horizontally, Ada's sideways shift at step 1 (further left means left); vertically, at step 2 (higher means right). Gray dots are actions from the demonstrations: two clusters, “both steps left” and “both steps right”. Blue bars along the bottom and the left are the bin tables: 16 intervals per step and their frequencies. Red hatched corners are combinations that never occurred in the demonstrations: step 1 one way, step 2 the other. Green and red dots are 100 actions from the model.

Bins over separate dimensions lose the links between them; picking one at a time restores the links at the cost of slow inference. A mixture of Gaussians describes each mode as a whole at once, with all its dimensions. Will Ada get past the kettle with it, and how is it different from diffusion?

Under the hood: mixture, energy and bins

Mixture of Gaussians. $p(a \mid s) = \sum_k \pi_k(s)\, \mathcal{N}\bigl(a;\, \mu_k(s), \sigma_k(s)\bigr)$ ; training minimizes $-\log p$ .
Energy. $\hat a = \argmin_a E(s, a)$ ; the energy $E$ is trained with InfoNCE on negative examples.
Bins. $p(a \mid s) = \prod_i p(a_i \mid s)$ : each dimension separately.
Autoregression. $p(a \mid s) = \prod_i p(a_i \mid s, a_1, \ldots, a_{i-1})$ : each next dimension sees the previous ones.

In RT-1 each of the 11 dimensions has 256 uniform bins, without autoregression. RT-2 and OpenVLA decode the dimensions one at a time, like words in a text. FAST compresses an action chunk with DCT and BPE and catches up with the diffusion-based π0 while training up to 5 times faster, but inference takes about 750 ms per chunk versus 100 ms.

In IBC the implicit model keeps a discontinuity sharp, while the explicit one, in the authors' words, “interpolates across each discontinuity”. The cost is unstable training: the Diffusion Policy authors call IBC “prone to training instability”.

In the interactive each step has 16 bins on the segment from −1.6 to 1.6 (in shift units). The data is 400 actions around “both steps left” and “both steps right” with a spread of 0.15.

🔬 Lab 2

Three Adas and a kettle

Let's check it on the scene. Regression, a mixture of two Gaussians and diffusion learn from the same demonstrations and drive to the same kettle at the same time. Regression and the mixture are real small networks that train right now in your browser. The mixture outputs two components, a plan for a detour on the left and a plan for a detour on the right, 24 steps ahead, and at the start of the drive it picks one of them. Diffusion is a model that starts from random noise and step by step turns it into a whole drive that resembles one of the demonstrations. Here it works without training: at every step the drive is corrected by an exact formula computed directly from the demonstrations. How diffusion works and how it is trained is covered in lessons 1.4 and 1.5.

Top view: Ada drives from the bottom up to the table, and the kettle stands on the straight route. Gray lines are demonstrations. Three Adas start side by side, each with its own policy and its own rim color: dashed is regression, dotted is the mixture, solid is diffusion. The thick line is the first run, the faint ones are the other nine, and a red cross is a kettle touch.

Training loss over the steps: MSE for regression, −log p for the mixture, that is, minus the logarithm of the probability the mixture gives to the actions from the demonstrations. Both curves should go down.

Left detour share50%

Steps between picks32 Demonstrations40 Mixture components2

regression mixture diffusion heat map

You can drag the kettle with the mouse or a finger.

A mixture that picks a side at every step jumps between the sides. A mixture that picks once per drive holds the side, but then it won't notice anything that happens on the way until the drive is over. We need something in between: pick the side neither at every step nor once per drive, but several steps ahead. How is such a plan built, and what happens where one plan gives way to the next?

Under the hood: three policies

Common to all three. All three policies see one number, the fraction of the path covered, and output the sideways offset from the route. A controller steers Ada toward it at no more than 9 cm per step. A touch means the centers come closer than 31 cm. A side switch counts when Ada crosses from one side of the kettle to the other, moving more than 6 cm away from its center. We count from one meter before the kettle until the kettle is behind.

Regression. A 1 → 32 → 32 → 1 network, tanh, MSE, 1200 Adam steps with batches of 32 examples.

Mixture of Gaussians. A 1 → 32 → 32 → 98 network outputs, at every step, $K$ offset plans 24 steps ahead, their probabilities $\pi_k$ and widths $\sigma_k$ . Training uses the negative log-likelihood, 1600 Adam steps. In the “every few steps” mode Ada picks a plan with probability $\pi_k$ and follows it for $H$ steps, then picks again; with $H = 32$ the second pick falls on the stretch past the kettle. In the “every step” mode a plan is picked anew at every step, and only its first action is executed. The width $\sigma_k$ isn't used at execution: Ada follows the center of the chosen component.

Diffusion. It generates the whole drive at once: 27 points of sideways offset, one every 2 steps. DDIM with 30 steps and a cosine noise schedule. Its denoiser is exact: for a noisy drive it returns a weighted mean of the demonstrations, with weights given by the noise likelihood. No network is trained here, so this diffusion can only repeat the demonstrations. We'll train a real denoiser in lesson 1.5.

Runs. Each launch is 10 runs: Ada's start is shifted slightly, and at every step about 0.5 cm of controller noise is added. Pressing again gives the next 10 runs with other seeds. The demonstrations don't change as long as the settings stay the same.

5 Action chunks

Action chunks and their seams

An action chunk is a set of actions several steps ahead that a policy outputs in one call: not one offset for the next step but a plan for $H$ steps. The robot executes the chunk's actions one after another without asking the network, and only when the chunk runs out does it call the policy again. The step where one chunk ends and the next one begins is called a seam, or chunk boundary. In the diagram one policy call gives 8 actions, Ada drives through them, and at the seam there's a new call.

On the left is the policy, on the right the time axis in drive steps and the sideways shift. Each policy call (the frame lights up) produces 8 orange dots on the axis: an action chunk 8 steps ahead. Ada (the circle with eyes) executes them one at a time; the dashed line is a seam, where one chunk ends and the policy is called for the next one. Click the diagram to pause it.

The mixture of Gaussians in the lab above was already outputting chunks: a plan 24 steps long, with the side picked once every 32 steps, so there wasn't a single seam before the kettle, which is at step 18. Let's shorten the chunk. Inside a chunk the side doesn't change, but at every seam the mixture picks again and may choose the other side. On the scene Ada's path is colored by the chosen side, and the dots on it are seams.

🔬 Lab 3

Top view: Ada drives from the bottom up to the table, the kettle is on the straight route, gray lines are demonstrations. The path of the first run is colored by the mixture component the chunk picked: turquoise is a detour on the left, purple on the right; white dots with an outline on the path are chunk seams. Faint lines are the other nine runs, and a red cross is a kettle touch.

Chunk, steps4

A long chunk holds the side but responds more slowly to what's going on: Ada notices anything that happens inside the chunk only at the next seam. We want a short chunk, and at the seam we want the new chunk to continue the old one. Chunks by themselves don't promise that: each one is an independent sample, and nothing makes it continue the previous one. The Real-Time Chunking paper (RTC; a Physical Intelligence method, 2025; lesson 1.5) says as much: neighboring chunks “may jump between different modes”. There are two ways to tie neighboring chunks together, and they work differently.

Smoothing. Chunks are output with overlap: a new one comes out before the old one has run out. At every step the robot executes not the latest chunk but a weighted mean of all the chunks that predicted this step; older ones weigh more, so the new one can't turn sharply. This is how ACT smooths seams; in lesson 1.7 you'll meet this trick under the name temporal ensembling. Consistency. At the seam several new chunks are sampled instead of one, and the robot executes the one closest to continuing the previous chunk (this is how the Bidirectional Decoding paper, 2024, does it). The diagram shows both methods at the same seam: the old chunk goes around on the left.

Two identical “time × offset” plots: horizontally the drive steps, vertically the offset from the straight route in centimeters (up is right, down is left); the hatched rectangle is the kettle. Turquoise chunk 1 goes around on the left; at the seam, step 8, chunk 2 appears, and the black line is what Ada executes. Smoothing: chunk 2 has picked the right side (purple), the mean of the two chunks is executed, and it runs between them, through the kettle. Consistency: four samples at the seam (dashed), the one closest to continuing chunk 1 is executed, and Ada stays on the left. Click the diagram to pause it.

Compare both methods on the same plot, but with real runs: a 16-step chunk, a new one every 4 steps, and the mixture picks the side of each one 50/50.

🧩 Interactive

This is not a top view but a plot, like the diagram above: horizontally, the steps of the drive to the table; vertically, the sideways offset from the straight route in centimeters (up is right). Vertical dashed lines are seams: a new chunk comes out every 4 steps. The hatched rectangle is the kettle: the steps at which Ada is level with it and the offsets at which she clips it.

Consistency holds the side but pays for every decision with several samples. Smoothing is cheaper but averages the modes just like MSE. We've learned to pick a side and hold it. But a mixture of Gaussians needs to know the number of modes in advance, and diffusion in the lab relied on an exact denoiser computed directly from the demonstrations. Which model can do without these limitations?

Under the hood: the seam scheme

Smoothing averages the overlapping chunks with weights, where $i = 0$ is the oldest chunk:

a_t = \frac{\sum_i w_i\, a_t^{(i)}}{\sum_i w_i}, \qquad w_i = e^{-m i}.

Consistency with $N$ samples takes the one closest to the remainder of the previous chunk.

A chunk is 16 steps long and a new one comes every 4 steps, so up to four chunks overlap at every step. The side of each new chunk is picked by a mixture of two Gaussians, 50/50. Ada moves sideways at no more than 9 cm per step. For smoothing $m = 0.5$ : however hard the new chunk tries to turn, the old ones weigh more. For consistency there are 8 samples, so the network is called 8 times as often.

The RTC authors solve the problem a third way: they freeze the steps that are already being executed and fill in the rest so that the new chunk continues the old one. We'll cover it in lesson 1.5, and the full picture of latencies and seams in lesson 1.7.

6 Check yourself

Eight common misconceptions

7 Takeaways

Lesson takeaways

The MSE minimum is the conditional mean. The mean of two correct actions can be wrong, and more data doesn't fix that: the network just hits the middle more precisely.
L1 gives the median: when one side outweighs the other, it picks that side; with an even split the solution is not unique.
Multimodality comes from the data: different operators, subtask order, pauses, spread within one operator, things the cameras don't see. Standard benchmarks have almost no conditional modes.
A distribution of actions is modeled with a mixture of Gaussians, bins and tokens, energy, a latent style, diffusion. Bins over separate dimensions lose the links between dimensions; picking one at a time brings them back.
Sampling at every step jumps between modes. An action chunk, a plan several steps ahead from one policy call, holds the side within itself; Ada can switch sides only at a chunk seam.
Chunk seams are made consistent, not averaged: the mean of chunks from different modes leads into the kettle again, while picking the sample closest to the previous chunk holds the side at the cost of several samples.

❓

One problem remains: a mixture of Gaussians needs to know the number of modes in advance, bins lose the links between dimensions, and diffusion in the lab worked with an exact denoiser that only repeats the demonstrations. We need a model that learns from data to sample from a distribution of any shape, so that every sample is plausible.

Open questions

Six operators record the same task in different ways. How do you check whether the model needs multimodality before choosing between regression with chunks and diffusion?

You can estimate the number of conditional modes, as Mazza et al. did: for similar observations, compare the continuations of the demonstrations and count how many of them are consistently different. A simpler and cruder way is to train both policies on part of the data and compare them on the same starts. If regression with chunks is no worse, the data is most likely unimodal. If it fails where the operators diverge, you need a generative head.

Why can a task be multimodal for the policy but not for the operator who demonstrated it?

The operator sees more than the robot: a different viewpoint, sounds, an agreement with a colleague, their own fatigue. Their choice is determined by what they know, but that information isn't available to the policy, and for the policy identical observations lead to different actions.

Paper guide

P. Florence et al., “Implicit Behavioral Cloning”, CoRL 2021.

Read closelyThe experiment fitting a discontinuous function: the explicit model “interpolates across each discontinuity”, while the implicit one keeps the discontinuity. And the tables with the real xArm6 arm: where energy beats MSE by the widest margin.
Safe to skipThe proofs that implicit models can approximate multivalued and discontinuous functions.
Question to keep in mindTo get an action, IBC searches for the energy minimum by trying candidates. Why is that more expensive than a single regression pass, and what happens to inference when the action has 7 dimensions?

Resources

Implicit Behavioral CloningThe IBC website: insertion videos with MSE and with energy
Behavior TransformersN. M. Shafiullah et al., 2022: a point goes around an obstacle on the left or on the right, clusters and offset, and CARLA with 98% right turns
robomimicA. Mandlekar et al., 2021: six operators with 50 demonstrations each, and a mixture of Gaussians versus a single action
Bidirectional DecodingY. Liu et al., 2024: making the new chunk consistent with the previous one
Real-Time ChunkingK. Black et al., 2025: chunk seams and why they aren't averaged; more in lesson 1.5
Understanding Multimodality in Generative BCMazza et al., 2026: estimating the number of conditional modes $\hat K$

➡️

Next: diffusion models. How do you sample from a complex distribution so that every sample is plausible? A diffusion model learns to remove noise from data and generates the other way round: it starts from pure noise and step by step moves toward the data. That's the subject of lesson 1.4, “Diffusion models”.

Was the lesson clear?

Found a mistake or inaccuracy? Report it on GitHub

Found this lesson useful? Say thanks — that is how the author knows the course is being read.