3.5 Part 3 · Foundation models for robots

Octo and OpenVLA: open models you can fine-tune yourself

In lesson 3.4, RT-2-X, a 55B-parameter model trained on a mixture of data from nine robots in Open X-Embodiment, outperformed models trained on a single robot's data. But its weights aren't publicly available, so you can't fine-tune it for your own arm and your own camera. In 2024, two open models trained on the same Open X-Embodiment data came out. Octo is a 93M-parameter transformer trained on 800,000 episodes: you can add a new camera or a new way of controlling the robot to it, and it fine-tunes on about a hundred demonstrations in a few hours on a single GPU. OpenVLA is a 7B-parameter Llama 2 language model with a vision encoder, trained on 970,000 episodes: on 29 tasks, its success rate was 16.5 percentage points higher than RT-2-X's. It can be fine-tuned by changing 1.4% of its parameters and stored at 4 bits per number with almost no loss. This lesson covers how both models are built and why a small update to the weights works as well as fine-tuning all of them.

≈ 45 minutes2 labs, an interactive and a quizmissions with a goal

FINE-TUNINGTransfer a small pretrained model to a new arm with a different camera in three ways and compare them with 5 and with 40 recordings

ACTION HEADAsk three models where to put a cube when there are two identical bowls on the table, and see which one chooses sometimes one bowl, sometimes the other

TABLESFind in the OpenVLA results where the closed 55B-parameter model is still stronger than the open one

Start the lesson ↓

1 Problem

The best model exists, but you can't fine-tune it yourself

In lesson 3.4, RT-1-X and RT-2-X were trained on a mixture of data from nine robots in Open X-Embodiment, and the 55B-parameter RT-2-X handled other robots' tasks better than models trained on a single robot's data. Now picture a lab with its own arm: the camera is mounted differently, the lighting is different, the objects are different, and there are only a few dozen recordings from this arm. A ready-made model trained on other robots makes mistakes on such a setup, and it has to be fine-tuned: trained further on your own recordings, starting from the ready weights rather than random ones.

You can't do that with RT-2-X. The model's weights haven't been released; you can only send it queries. The OpenVLA authors, who compared their model with RT-2-X, state plainly that “fine-tuning of RT-2-X is not supported through its inference API.” The open RT-1-X is small, at 35M parameters, and in lesson 3.4 it couldn't cope with the large domains, while in RT-2 the knowledge from the internet came precisely from a large vision-language model (VLM, lesson 3.2).

What you need is a model with open weights, training code and data, trained on the same Open X-Embodiment mixture, that can be fine-tuned in hours on a single GPU rather than on a cluster. In 2024, two such models came out, with different answers to the question of where to start. Octo (UC Berkeley, Stanford, CMU and Google DeepMind) is a 93M-parameter transformer trained from scratch on robot data. OpenVLA (Stanford, UC Berkeley, Toyota Research Institute and others) is a 7B-parameter vision-language model that, like RT-2, was fine-tuned to output actions.

The robots in Open X-Embodiment differ: some have a wrist camera, others only a side camera; some get the command in words, others as a goal image; some control a gripper, others their joints. What does a model need so that it can be transferred to any of them?

2 Octo

Octo: inputs and outputs are added as tokens

RT-1-X takes one image and a command and outputs 7 numbers. If a new arm has a wrist camera, a force sensor or different joints, such a model has to be rebuilt and large parts of it retrained. In Octo, everything the model takes in and puts out is tokens, and a new input or output is added as new tokens.

Each kind of input has its own tokenizer. The command is read by the pretrained 111M-parameter t5-base language model, which outputs 16 tokens. The image goes through a few convolutional layers and is cut into 16 × 16-pixel patches: the side camera yields 256 tokens, the wrist camera 64. Instead of a command, you can give a goal image, a picture of what the scene should look like at the end; it's turned into tokens the same way. The backbone of Octo-Base is a 12-layer transformer of width 768, 93M parameters in total; there is also the 27M-parameter Octo-Small.

Attention in the backbone is restricted by masks: observation tokens see the task tokens and the observation tokens of the same and earlier steps, but not of later ones. The sequence also gets readout tokens: learned tokens that read all the others but are read by none, so they don't affect the other tokens. They work like the [CLS] token in BERT, except that there's one per time step. Their output goes to the action head, a small network on top of the backbone that turns the backbone's output into action numbers. So to add a force sensor, it's enough to add its tokenizer, and to control new joints, a new head: the backbone weights don't change.

Octo's head is a diffusion head, like Diffusion Policy in lesson 1.5: a small three-layer network of width 256 turns random noise into an action chunk for several control steps ahead in 20 denoising steps. The backbone runs once per prediction, and all 20 denoising steps happen in the head alone, which is cheap. The authors compared three heads on Octo-Small: with action tokens, the model succeeded in 18% of trials, with mean-squared-error regression in 35%, and with diffusion in 83%. How the heads differ is the subject of Lab 2.

Octo was trained on 25 Open X-Embodiment datasets, 800,000 episodes; RT-X had 350,000. Datasets with more diverse tasks and scenes got double weight. Training Octo-Base took 14 hours on a TPU v4-128 pod. Without fine-tuning, Octo had on average a 29% higher success rate than RT-1-X, and on the WidowX and Google robots it performed about as well as RT-2-X.

The main test is fine-tuning. In six new setups, including one with a force sensor, one with joint control and the bimanual ALOHA with 14 numbers per action, the authors took about 100 demonstrations and the same settings everywhere: 50,000 steps, full fine-tuning of all the weights. One such run takes about 5 hours on a single NVIDIA A5000 GPU with 24 GB of memory. On average, the fine-tuned Octo succeeded in 72% of trials, a policy trained from scratch on the same demonstrations in 20%, and a model with the pretrained VC-1 vision encoder in 15%. The from-scratch policy wasn't Octo itself but a 28M-parameter ResNet with a small transformer: Octo's large transformer overfit quickly on small datasets.

But Octo's backbone has seen only robot data. In RT-2, knowledge of objects and words came from a model trained on images and text from the internet. Can you build an open model on such a foundation?

3 OpenVLA

OpenVLA: an open vision-language model outputs actions as tokens

RT-2 in lesson 3.3 showed that a vision-language model allowed to write actions as tokens carries knowledge from the internet over to the robot: it picks up an object by its name even if that object wasn't in the robot data. But both RT-2 and RT-2-X are closed. OpenVLA repeats the recipe with open parts.

The backbone is the open Prismatic-7B model. A 224 × 224-pixel image is read by two vision encoders at once, 600M parameters together: SigLIP, trained to match images with captions, knows well what is in the picture, while DINOv2, trained on images without captions, is better at conveying where things are. The features of each image patch from the two encoders are concatenated into one vector. A two-layer projector network maps these vectors into the word space of the 7B-parameter Llama 2 language model, as in lesson 3.2.

The action is 7 numbers: the displacement and rotation of the gripper and its opening. Each number is split into 256 bins, as in RT-2, but the bounds are set between the 1st and 99th percentiles of the values in the data rather than between the minimum and the maximum: otherwise rare outliers would stretch the scale and make the bins coarser. The 256 least used tokens of the Llama vocabulary are overwritten to stand for the 256 bins. The model learns to predict the next token, and the loss is computed only on the action tokens.

The data is 970,000 Open X-Embodiment episodes with mixture weights like Octo's. Training took 14 days on 64 A100 GPUs, 21,500 A100-hours, and 27 epochs over the data: the model kept improving until its accuracy on the action tokens exceeded 95%. The vision encoders were fine-tuned together with the rest of the model. In vision-language models the encoder is often frozen: LLaVA does this, and according to the Prismatic authors, it usually works better. PaliGemma from lesson 3.2, by contrast, freezes nothing. For actions, freezing turned out worse: with frozen vision, the results were noticeably lower. The authors hypothesize that the features learned from the internet lack the fine-grained details needed for precise motion. The trained model needs 15 GB of GPU memory to run and outputs about 6 actions per second on an RTX 4090.

On 29 tasks on two robots, OpenVLA's success rate was 16.5 percentage points higher than RT-2-X's, even though it has 7 times fewer parameters; the breakdown by category is in the interactive below. Fine-tuning worked too: on seven tasks with a Franka arm, with 10–150 demonstrations each, the fine-tuned OpenVLA succeeded in 67.2% of trials on average on the tabletop setup, Diffusion Policy trained from scratch in 48.5%, and the fine-tuned Octo in 43.4%.

But this is fine-tuning of all 7 billion weights: each task took 5 to 15 hours on eight A100s. How do you fine-tune such a model on a single GPU?

4 LoRA

LoRA: train a small update, not the whole matrix

When all the weights are fine-tuned, the GPU holds not only the 7 billion weights but also a gradient for each of them and the two moments of the Adam optimizer. For OpenVLA with a batch of 16 examples, that's 163.3 GB, which doesn't fit on one GPU, so the model was split across two. You can train only some of the weights and freeze the rest.

The OpenVLA authors compared five approaches on two tasks with a Franka arm. Only the last layer and the token embedding matrix: 30.3% successful trials. Frozen vision, with everything else trained: 47.0%. The “sandwich”, where the vision encoder, the embedding matrix and the last layer are trained: 62.1%. All the weights: 69.7%. And LoRA: 68.2%, almost as good as all the weights, even though only 1.4% of the parameters are trained.

LoRA (low-rank adaptation, Hu et al., 2021) keeps every weight matrix $W$ frozen and trains an update made of two narrow matrices next to it. The layer computes $W x + B A x$ : the matrix $A$ compresses the input to $r$ numbers, and the matrix $B$ expands them back to the size of the output. The number $r$ is the rank of the update: the matrix $B A$ has at most $r$ linearly independent rows, so it changes the layer's behavior in only $r$ directions. At the start, $B$ is filled with zeros, so the model is identical to the pretrained one, and after training the update can be added to $W$ , so the model runs as fast as the original.

A diagram of one layer, not a drawing of the model. Gray with a lock is the frozen matrix $W$ , orange marks the trainable matrices $A$ and $B$ , blue dots are the input and output numbers, and red dots are the gradient during training: it reaches only $A$ and $B$ . The numbers at the bottom are for the first layer of our mini-model from Lab 1. Click the diagram to pause it.

For a matrix of size $d_\text{out} \times d_\text{in}$ , you normally train $d_\text{out} \cdot d_\text{in}$ numbers, while the update has $r\,(d_\text{in} + d_\text{out})$ . For a large square matrix with side $d$ , that's a fraction of $2r/d$ : the larger the matrix, the smaller the fraction. In OpenVLA, rank-32 LoRA was applied to all linear layers: 97.6M numbers out of 7,188.1M are trained, the memory needed is 59.7 GB instead of 163.3 GB, and fine-tuning takes 10–15 hours on a single A100, 8 times less compute than full fine-tuning. Rank 64 has twice as many parameters but the same success, 68.2%, so the authors recommend rank 32.

To run on the robot, the model can also be compressed. Quantization stores each weight in 8 or 4 bits instead of 16. With 4 bits, OpenVLA succeeded in 71.9% of trials on eight tasks versus 71.3% at 16 bits, and took 7.0 GB of memory instead of 16.8 GB. At 8 bits, success dropped to 58.1%, but because of speed, not accuracy: on an A5000 GPU, this model output 1.2 actions per second, with a controller designed for 5. When the robot waited for each action, all three variants performed about the same.

Why does a low-rank update fine-tune as well as all the weights, while a frozen backbone does worse? And is it always so, say, with 5 recordings instead of 50? Let's check on the table from lesson 2.1.

🔬 Lab 1

Fine-tuning for a new arm: all weights, LoRA or freezing

Let's transfer a small pretrained model to a new arm. The model is pretrained on the robot from lesson 2.1: the table seen from above, cubes of three colors, an ordinary camera. The new arm has different objects (cans instead of cubes) and a different camera: the light is warmer and the blue channel is weaker. Without fine-tuning, the model picks up the named object in only 28% of new scenes. The new arm's recordings are made by an operator, a person who guides the arm and shows what to pick up. The operator records in series: four recordings in a row with one command, then four with the next.

The model is built like a small VLA. The table is a 24 × 24 grid of cells, as in lesson 2.1. For each cell, the model looks at a 3 × 3 window of neighboring cells and at the cell's coordinates, and the command word (“red”, “blue” or “green”) is added to the first layer as a shift vector. Two hidden layers (40 and 20 neurons) output a score for the cell, and a softmax over the 576 cells gives where to pick. That's 2,281 parameters in total. Pretraining runs right on the page: 60 recordings from the robot of lesson 2.1, 400 Adam steps. On new scenes of the same robot, the model picks up the named cube 95% of the time.

There are three fine-tuning approaches, each with 300 training steps on the new arm's recordings. All weights: everything is trained. LoRA: the weights are frozen, matrices $B A$ of rank $r$ are added to the matrices of both layers, and the last layer is trained as well. Freezing: the whole backbone is frozen, and on top of it sits a new head with a hidden layer of 8 neurons. The test is 60 new scenes of the new arm, 20 for each command. All the numbers are computed in your browser.

A top view of the table, as the camera sees it. On the left is a scene of the robot the model was pretrained on, on the right a scene of the new arm. The ring marks the object named in the command.

Pretraining on the robot from lesson 2.1…

A top view of the new arm's table. The same new scene for all models: the ring is the can from the command, the orange shading shows how confident the model is that it should pick here, and the cross is the cell the model chose: green if it hit the right can, red if not.

scene 1 of 60

Recordings

LoRA rank

set 4

Under the hood: the mini-model, the three approaches and why it turned out this way

The features of cell $i$ are the 27 color numbers of the 3 × 3 window minus the background color (the median over the scene), plus the two coordinates of the cell. Let $x_i$ be these 29 numbers and $e_c$ the vector of the command word $c$ . The model computes

\begin{aligned} h_1 &= \tanh\bigl(W_1 x_i + b_1 + e_c\bigr), \\ h_2 &= \tanh\bigl(W_2 h_1 + b_2\bigr), \\ s_i &= w_3^{\top} h_2 + b_3, \\ p(i) &= \operatorname{softmax}_i (s_i), \end{aligned}

and the loss is the cross-entropy at the cell where the operator picked up the object. The matrices: $W_1$ is 40 × 29, $W_2$ is 20 × 40. LoRA replaces them with $W_1 + B_1 A_1$ and $W_2 + B_2 A_2$ ; at rank 2, the updates are $2 \cdot (29 + 40) + 2 \cdot (40 + 20) = 258$ numbers, 279 with the last layer, that is, 12% of the parameters. The fraction is large here because the matrices are small: with $d \approx 40$ , the ratio $2r/d$ at rank 2 is about 10%, while for OpenVLA's matrices it's 1.4% at rank 32. With freezing, a new head is trained: $s_i = w_o^{\top} \tanh(W_h h_2 + b_h) + b_o$ , which is 177 numbers.

Without fine-tuning, the model on the new arm never picked up the red can out of 20 tries, picked the blue one 13 times and the green one 4 times: the warm camera shifted the colors, and the color features learned with the old camera no longer match the new ones. The five recordings have four “red” commands and one “blue”, and no “green”. All weights and LoRA change the color features themselves in the first layer and can fit them to the command that dominates the recordings, spoiling the others. Freezing touches neither the features nor the words and learns only how to read the backbone's output: it does less damage to the commands that are almost absent from the recordings, but it also can't fix the features that the new camera spoiled. Look at the per-command bars under the scene.

With five recordings, the result depends on which scenes they happen to contain: the “Another set” button takes different recordings. We checked five sets: on average, with 5 recordings, freezing succeeded in 62% of trials, LoRA and all weights in 48% each; with 40 recordings, all weights reached 91%, LoRA 90% and freezing 84%. The Octo and OpenVLA papers didn't try so few recordings: Octo used about 100 per task, OpenVLA 10–150, and there full fine-tuning and LoRA beat freezing.

With a few dozen recordings, a low-rank update fine-tunes the model as well as all the weights do: the change needed for the new arm fits into a few directions. A frozen backbone is robust when there are very few recordings, but it has a limit: it can't fix the features that the new camera spoiled. Something similar shows up in the OpenVLA table: the variants where the image features don't adapt to the new scene, last layer only (30.3%) and frozen vision (47.0%), fell well behind LoRA (68.2%).

But the model learns more than seeing. Its head decides which exact action to output, and Octo and OpenVLA have different heads: Octo uses diffusion, OpenVLA uses tokens. What difference does this make when there are several correct actions?

🔬 Lab 2

The action head: the mean, tokens or diffusion

Often there are several correct actions. In lesson 1.3, the teacher drove Ada around the kettle sometimes on the left and sometimes on the right, and a policy trained by regression with mean squared error drove straight into the kettle: the average of two correct actions turned out to be wrong. Octo and OpenVLA have different action heads, and it's the head that decides what to do with several correct answers.

Ada is a home humanoid on a wheeled base; the course labs are built around her. More about her in lesson 0.1.

The task: the gripper holds a cube above the table, there are two identical bowls on the table, and the cube has to go into either of them. The operators recorded 40 demonstrations: 21 times they put the cube into the top-left bowl, 19 times into the bottom-right one. The scene is always the same, so the backbone's output is the same for all the heads, and the only difference is the head.

Regression outputs two numbers, a point on the table, and learns with mean squared error, like the policies of lesson 1.1. Tokens: the table is split into bins along each axis, as in RT-2 and OpenVLA; the model outputs the token for the x coordinate first, then the y token given the chosen x, and we take the most likely token; the answer is the center of the bin. Diffusion works as in Octo: starting from a random point, a small network removes the noise in 20 denoising steps and arrives at an action. All three are trained right here: regression by gradient descent, tokens from the frequencies in the recordings (a classifier with cross-entropy converges exactly to them), and the 5 → 64 → 64 → 2 diffusion network in 2,000 Adam steps.

A top view of the table, 96 × 96 cells of the fine grid from lesson 2.1. The two bowls are two correct targets, and the gray dots are where the operators put the cube. The heads' answers: square for regression, diamond for tokens, circle for diffusion; the dashed line is the path of the last diffusion answer from noise to action.

Legend: marks on the scene

Bins per axis24 Into the left bowl21 of 40

Under the hood: three heads, bins and the denoising step

Regression minimizes the mean squared error $\tfrac1N \sum_j \lVert a - a_j \rVert^2$ , and its optimum is the mean of the recordings, $\bar a = \tfrac1N \sum_j a_j$ : a point between the bowls.

Tokens. The axis is split into $K$ bins, each $96/K$ cells wide. The model outputs a distribution $p(x)$ , then $p(y \mid x)$ , the way a language model outputs words one after another. A classifier trained with cross-entropy converges to the frequencies in the data, so here the probabilities are the frequencies of the recordings. The answer is the most likely token on each axis, so it's always the same; its precision is limited by the bin width. If you sample the tokens at random according to their probabilities, the answers will also split between the two bowls, but the precision will stay the same.

Diffusion. The network $\varepsilon_\theta$ takes the noisy action and the denoising step index $k$ and predicts where the clean action is; the noise is expressed through this prediction. A denoising step is formula (1) of the Octo paper:

x_{k-1} = \alpha\bigl(x_k - \gamma\, \varepsilon_\theta(x_k, e, k) + \mathcal N(0, \sigma^2 I)\bigr),

where $e$ is the backbone's output, and $\alpha, \gamma, \sigma$ are set by the noise schedule; here, as in Octo, it's a cosine schedule with 20 denoising steps. Our network is small and trains in a couple of seconds, so about 7 answers out of 100 land outside the bowls.

Regression picks an average that isn't in the data and puts the cube outside both bowls. Tokens aren't confused by two correct answers: the cube lands in a bowl, just always the same one. The weakness of tokens lies elsewhere. The answer is the center of a bin, and it's never more precise than the bin width. And every number of the action is a separate token, which OpenVLA outputs one after another, so it manages only about six actions per second. Diffusion outputs sometimes one bowl, sometimes the other, in roughly the same proportion as the operators did, and its answer isn't tied to bins.

Octo's ablations on the WidowX robot show the same thing on real tasks: with the diffusion head, the model succeeded in 83% of trials, with regression in 35% and with tokens in 18%. The authors explain the difference this way: the diffusion head “can model multi-modal action distributions (unlike the MSE head) while retaining the precision of continuous actions (unlike the discrete head).” The regression policy was “hedging”: it moved very slowly and, for example, failed to rotate the gripper. The token policy, with 256 bins per axis, was more “decisive”, but it lacked precision and often missed the grasp. What let Octo down with tokens was precision, not the choice between correct actions. Yet OpenVLA, which also uses tokens, is stronger than Octo on the same robots. So what decides the results?

🧩 Interactive

The OpenVLA tables: where the open model beats the closed one

Comparing models by a single number is risky: tasks differ. The OpenVLA authors tested four models without fine-tuning on two robots. On WidowX with the BridgeData V2 data: 17 tasks with 10 trials each, grouped by what changed compared with the training data. On the Google mobile manipulator, the robot on which the RT-1 data was recorded: 12 tasks with 5 trials each, in familiar and in new conditions. The scenes and starting positions are the same for all models. Sometimes half a trial was credited, for example when the arm went for the right object but didn't finish the job.

The table shows the share of successful trials. Hover over a row or click it to see the tasks and each model's number of successes.

The numbers come from Tables 4 and 6 of the OpenVLA paper: the share of successful trials in percent, and the bar shows the same share. Bold marks the best in the row. Under each model's name: its number of parameters and whether its weights are open.

The authors attribute the lead over RT-2-X to three causes: more data (970,000 episodes versus 350,000), cleaner data (the first step, where the action is zero, was removed from every BridgeData V2 recording; otherwise the model learned to stand still) and dual vision. They checked the data and the second encoder on eight tasks: OpenVLA succeeded in 76.3% of trials, the same model trained only on BridgeData V2 in 45.6%, and that model without DINOv2 as well in 40.6%. The Open X-Embodiment data contributed the most.

RT-2-X is stronger only where knowledge from the internet is needed: it was co-fine-tuned on robot actions together with web data, while OpenVLA was fine-tuned on robot data alone. In these tests, Octo and RT-1-X often picked up the wrong object when other objects lay nearby, and the OpenVLA authors deliberately made the tests harder than in earlier papers. What of the 2024 open models is still in use in 2026?

5 Today

What grew out of the open VLAs

In February 2025, the OpenVLA authors proposed a new recipe for fine-tuning it, OpenVLA-OFT (Kim, Finn, Liang). Instead of tokens, the model outputs continuous actions trained by regression with an L1 loss, the whole action chunk at once rather than token by token. On the LIBERO simulation benchmarks, the model fine-tuned this way succeeded in 97.1% of trials on average across four task suites; in the OpenVLA paper, the model fine-tuned with LoRA reached 76.5% on the same suites. Actions are generated 26 times faster.

Fine-tuning on a single GPU has become routine. Hugging Face's LeRobot library records demonstrations on inexpensive arms such as SO-101 and Koch, and trains and runs policies: ACT, Diffusion Policy, π0 and SmolVLA. SmolVLA (June 3, 2025) is a 450M-parameter model based on SmolVLM2, pretrained on 487 datasets recorded by members of the LeRobot community, fewer than 30,000 episodes. It trains on a single consumer gaming GPU or even on a MacBook. On tasks with the SO-100 arm, pretraining on community data raised its success rate from 51.7% to 78.3%.

Companies release open weights too. NVIDIA published the 3B-parameter GR00T N1.6 for humanoids on Hugging Face (the model card dates from December 2025; NVIDIA license with restrictions on use): a vision-language model reads the image and the command, and a flow-matching transformer outputs continuous actions. Physical Intelligence released the weights and code of π0, π0-FAST and π0.5 under the Apache 2.0 license in its openpi repository, adding π0.5 and a PyTorch version in September 2025. LoRA fine-tuning there needs more than 22.5 GB of memory, which means an RTX 4090, and full fine-tuning more than 70 GB, an A100 or H100. It's the same arithmetic as for OpenVLA: LoRA fits on one GPU, full fine-tuning doesn't.

OpenVLA-OFT, GR00T N1.6 and π0 output continuous actions, not tokens; π0-FAST, on the contrary, compresses actions into tokens, and lesson 3.6 covers it too. OpenVLA outputs its seven action tokens one after another, which on an RTX 4090 comes to about 6 actions per second. The authors themselves call this a key limitation: the bimanual ALOHA is controlled at 50 Hz. Tokens are also coarse: their precision is limited by the bin width. How do you keep the knowledge of a vision-language model but output precise actions quickly? That's the subject of lesson 3.6.

6 Check yourself

Eight common misconceptions

7 Takeaways

Lesson takeaways

RT-2-X is closed, and you can't fine-tune it for your own arm. Octo and OpenVLA are fully open (weights, training code and data) and trained on Open X-Embodiment: 800,000 and 970,000 episodes.
In Octo, everything is tokens: the command, the images, the goal. Readout tokens read the others without affecting them and pass their output to the head, so a new input or output is added without touching the backbone. About 100 demonstrations and 5 hours on a single GPU, and Octo works on a new setup.
OpenVLA is a 7B-parameter vision-language model with two vision encoders, SigLIP and DINOv2, that outputs actions as tokens, 256 bins per axis. On 29 tasks, it is 16.5 percentage points stronger than RT-2-X, but weaker where knowledge from the internet is needed.
LoRA freezes the matrix $W$ and trains an update $B A$ of rank $r$ . In OpenVLA, that's 1.4% of the parameters and almost the same success as fine-tuning all the weights; the rank barely matters. Quantization to 4 bits cuts memory by more than half without a loss in success.
The head decides what to do when there are several correct actions, and how precisely. Regression outputs the mean; tokens with the most likely answer always give the same correct action, but only to within a bin and one token at a time; diffusion gives sometimes one, sometimes the other. In Octo's ablations, tokens got 18% versus 83% for diffusion: the policy often missed the grasp.
In our lab, the frozen backbone worked best with 5 recordings, while with 40, all weights and LoRA drew level with each other and overtook it: the low-rank update gets enough data, and freezing lacks freedom.

Open questions

Why is a rank-32 update to matrices with thousands of rows enough for a new setup?

The LoRA authors hypothesized that the weight change needed for a new task itself has a low rank: the model can already do almost everything, and the adaptation happens in a few directions. In OpenVLA, ranks 32 and 64 gave the same success, and in our lab, so did ranks 1 to 8. But in the OpenVLA experiments, the new setup is a Franka arm, the same as in the DROID dataset from the training mixture. If a new robot is very different, for example in its number of joints, like ALOHA for Octo, an update alone isn't enough: it needs a new head and possibly more data.

OpenVLA outputs about 6 actions per second. How do you speed it up to the 50 Hz control rate of the bimanual ALOHA?

The authors suggest predicting an action chunk in a single call, as Octo and ACT do, or speeding up token decoding with language-model techniques such as speculative decoding. OpenVLA-OFT outputs the whole chunk of continuous actions in one pass and generates actions 26 times faster. π0 from lesson 3.6 hands the actions over to an action expert, a second set of weights in the same transformer that builds the chunk with flow matching.

Paper guide

M. J. Kim, K. Pertsch, S. Karamcheti et al., “OpenVLA: An Open-Source Vision-Language-Action Model”, CoRL 2024.

Read closelySection 3.2: how actions become tokens and why the bin bounds are set by percentiles. Section 5.3 and Table 1: five fine-tuning approaches, their parameters and memory. Section 5.4 and Appendix D.4: why 8 bits turn out worse than 4.
Safe to skipAppendix B with descriptions of each of the 29 tasks and Appendix E with the tests in the LIBERO simulation.
Question to keep in mindAppendix C says that RT-2-X on BridgeData V2 often output a zero action and froze, so it was queried for its second most likely action. How does this change the comparison between the two models?

Octo Model Team, D. Ghosh, H. Walke, K. Pertsch et al., “Octo: An Open-Source Generalist Robot Policy”, RSS 2024: Section III-A on tokens and readouts, Table II and Appendix E on what worked and what didn't.

E. J. Hu et al., “LoRA: Low-Rank Adaptation of Large Language Models”, 2021: where the low-rank update comes from.

Resources

OctoProject page: videos and links to the code and weights
Octo codePretraining and fine-tuning in JAX, Open X-Embodiment data loaders
OpenVLAProject page: experiment videos and links to the weights
OpenVLA codeTraining in PyTorch, LoRA fine-tuning and 4-bit inference
OpenVLA-OFTA recipe for fast fine-tuning: continuous actions and action chunks
LeRobotHugging Face's library for recording demonstrations and training and running policies
SmolVLAHugging Face blog post on the 450M-parameter model
GR00T N1.6NVIDIA's model card on Hugging Face
openpiWeights and code for π0, π0-FAST and π0.5

➡️

Next: 3.6 “π0”. OpenVLA outputs an action as tokens: one at a time, about 6 times per second and only to within a bin. π0 keeps the vision-language model for understanding the scene and the command, while the actions are built by a second, smaller set of weights in the same transformer: a chunk of continuous actions via flow matching. Lesson 3.6 →

Was the lesson clear?

Found a mistake or inaccuracy? Report it on GitHub

Found this lesson useful? Say thanks — that is how the author knows the course is being read.