3 Part 3 · Foundation models for robots
Part 3 check
The quiz at the end of a lesson tests that lesson right after you read it. Here the tasks from all lessons of the part come shuffled: that shows what has stuck and whether the topics connect in your head. At the end you get a score per lesson and links to the sections worth revisiting.
3.1–3.2RT-1 and vision-language models: actions as tokens, an image inside a language model
3.3–3.4RT-2 and Open X-Embodiment: co-fine-tuning, one format for 22 robots
3.5–3.6Octo, OpenVLA and π0: open models, LoRA, action chunks from flow matching
3.7–3.8Dual-system architectures and π0.5–π0.7: two systems, subtasks in words, generalization
✓ Result
Check result
What to revisit
➡️
Next: Part 4, data for training robots. The models of Part 3 consume thousands of hours of demonstrations. Part 4 is about where to get them: teleoperation and data collection devices, datasets, human video, simulation and the data flywheel.