0.2 Part 0 · Introduction

Inside a robot

In Lesson 0.1 the network produced a command, and Ada’s gripper instantly moved exactly where it was told. In a real robot, commands are executed by motors with gearboxes, and the robot learns about itself and the world through noisy sensors. A policy is a neural network that picks the robot’s next action from what the robot sees. If you train it on a simplified model of the body, it will get a different response on the real robot. We’ll take the robot’s body apart piece by piece, from what the policy sees down to the motors, and at the end close the gap between a simulated actuator and a real one with a small neural network.

Ada is a home humanoid on a wheeled base; the course labs are built around her. More about her in Lesson 0.1.

≈ 40 minutes2 labs and 5 interactives

BODYWhat the policy receives and what it outputs: a humanoid diagram and four robots side by side

JOINTSReach the cup behind the box and find out why an arm needs a third joint

IMPACTPick a gearbox for a jumping robot’s knee: the motor needs torque, and the joint needs a soft landing

ACTUATORSSee how six actuators work and sort robot parts by actuator type

IMUTune a filter for the body tilt and rotation sensor (IMU) so that its tilt error stays under 1.6°

MODELFind where the simulator diverges from the actuator and close the gap with a network

Start the lesson ↓

1 Input and output

What the policy sees and what it controls

What does a policy take as input, and what does it output? A language model answers in text and can think about its reply for as long as it likes: the user will wait. A robot can’t do that. While the network is computing, Ada keeps driving, falling or squeezing the cup, and the motors need new commands thousands of times per second. A big network can’t keep up with that.

That’s why control is split into levels. A fast lower level drives the motors; a slow upper level decides what to do. The neural network usually sits in the middle: a few dozen times per second it outputs targets, such as joint angles. From there, controllers take over: simple programs that compare the target with sensor readings and compute how hard to drive the motor.

Click the robot’s parts: what’s inside, how fast it runs, and what the policy gets out of it. Below, the same questions for four well-known robots.

Four robots side by side

So all four policies receive joint angles, add cameras or an IMU, and output target angles that the controllers execute. The policy controls joints. How many does it take to reach a target?

2 Degrees of freedom

A joint is one degree of freedom

How many joints does an arm need? Each joint adds one degree of freedom: an angle you can change. A gripper on a plane has two coordinates, so it seems two joints should be enough. But a two-link arm reaches each point in only two poses: elbow up and elbow down. If both hit an obstacle, the target is out of reach.

The way out is to add a third joint. It doesn’t make the arm longer, but it lets you choose the pose without moving the gripper. Below are two tasks: drag the gripper and the elbow with your mouse or finger.

Collision-free workspace Lamp

Under the hood: how the arm chooses a pose

When you drag the gripper, the arm solves inverse kinematics: which joint angles put the gripper at the desired point. For two links of length $a$ and $b$ , the answer comes from the law of cosines, and there are always two solutions, with the elbow on either side of the shoulder → gripper line:

\begin{gathered} \cos q_2 = \frac{x^2 + y^2 - a^2 - b^2}{2ab},\\ q_2 = \pm\arccos(\ldots). \end{gathered}

With three links there are infinitely many solutions, so you need a selection rule. In the first task the scene keeps the hand vertical; in the others it keeps the hand angle from the start of the drag. If the target can’t be reached that way, it takes the closest feasible angle. When you drag the elbow, the gripper is pinned: the elbow moves along a circle around the shoulder, and the two remaining links solve the two-link problem to the gripper anew each time. That is motion in the null space.

Collisions are checked along the link axes: a link is treated as 4.4 cm thick, and the box, the lamp and the table as rectangles. The workspace is a grid of 2.5 cm cells where the gripper can be placed in at least one collision-free pose.

An extra joint gives you a choice of pose: the gripper stays put while the elbow moves away from the obstacle. But in this scene the joints snapped to the right angle on their own. In a real joint, a motor turns the link, and the main thing it provides is torque: the rotational force it turns the joint with. Torque is measured in newton-meters, N·m: 1 N·m is a force of 1 newton, the weight of about 100 g, on a 1 m lever. Does the motor have enough torque?

🔬 Lab 1

Gearbox and impact

A motor on its own doesn’t have the strength. It is fast but weak: in the lab it delivers 1.6 N·m, while the knee of a jumping robot needs at least 18.

The usual fix is to put a gearbox between the motor and the link. With gear ratio $N$ it multiplies torque by $N$ and divides speed by the same factor. It seems the bigger $N$ , the better.

Let’s check what a large gear ratio costs. On the scene are two identical joints with different gearboxes: 6:1, like Mini Cheetah, and 100:1, like a harmonic drive. The push and the impact hit both at once, so the difference shows on a single scene. Below the scene is a comparison table. Reflected rotor inertia shows how hard it is to spin up or stop the rotor (the rotating part of the motor) when pushing from the link side. Gearbox friction is the torque spent just on turning the gearbox itself.

Link A6 : 1 Link B100 : 1

Under the hood: what a gearbox does

A gearbox with ratio $N$ multiplies the motor’s torque and divides its speed. Rotor inertia "reflected" to the output grows as the square of $N$ :

\begin{aligned} \tau_{\text{out}} &= N\,\tau_{\text{motor}}, \\ \omega_{\text{out}} &= \omega_{\text{motor}} / N, \\ J_{\text{reflected}} &= N^2 J_{\text{rotor}}, \\ J_{\text{eff}} &= J_{\text{link}} + N^2 J_{\text{rotor}}. \end{aligned}

On impact with a rigid wall of stiffness $k$ at speed $v$ , the peak torque grows roughly as $v\sqrt{k\,J_{\text{eff}}}$ : the larger the effective inertia, the harder the hit. In MIT Cheetah 2, a rotor inertia of 3.03·10⁻⁴ kg·m² at a gear ratio of 5.8 gives about 0.01 kg·m² at the output. A direct drive with the same torque would need a motor 76 cm in diameter.

In the lab: rotor inertia 1.2·10⁻⁴ kg·m², peak motor torque 1.6 N·m, link inertia 0.09 kg·m². Gearbox friction grows with $N$ : Coulomb friction $0.02\,N$ N·m plus viscous. The wall is a spring with a damper; the link accelerates to 3 rad/s. When you drag the link, your hand acts as a spring between your finger and the end of the link, and the motor is off.

A small gear ratio has one more advantage: torque can be estimated from motor current: $\tau \approx N K_t\, i$ . That’s how MIT Cheetah gets by without force sensors in its legs. The price is estimation error: in Cheetah 3, gearbox friction adds about ±10%.

A large gear ratio gives torque, but reflected rotor inertia grows as its square. The actuator loses backdrivability: an external push barely turns the motor, and on impact the actuator hits harder. An actuator with a small gear ratio is backdrivable and hits softer, but gives little torque. A backdrivable actuator has one more property, transparency: force from the output reaches the motor almost undistorted, and the current tells you how hard the leg is pushing on the ground. How else do engineers look for a compromise?

3 Types of actuators

Six ways to move a joint

In the lab only the gear ratio changed. A large one gives torque, but the actuator barely yields to a push and hits harder on impact. With a small one the actuator is backdrivable, but the motor lacks torque. Engineers look for a compromise in other ways too: adding a spring, moving the motor out of the joint, turning rotation into linear motion, or using oil under pressure.

Below are six actuators found in well-known robots. The diagrams are simplified: they show the principle, not real dimensions. Orange marks what the motor turns, blue marks the link it moves. Click a diagram to pause it.

Quasi-direct drive

How it works: A large-diameter motor and a single gear stage, about 6 : 1. The motor itself produces high torque, so it doesn’t need much multiplication.
What it gives: The actuator is backdrivable: an outside push turns the motor, and the leg takes the impact softly. And transparent: force at the foot is estimated from motor current, with no separate sensor.
What it costs: For its torque the motor is large and heavy. If it holds a load for a long time, it heats up.
Where it’s used: The legs of MIT Mini Cheetah and Unitree robots.

Harmonic drive

How it works: An oval cam rotates inside a flexible toothed wheel and presses it against a rigid ring at two points. The wheel has two teeth fewer than the ring, so for each cam revolution it turns by just two teeth. The diagram shows a 30 : 1 ratio; real drives go from 30 to 160 in a single stage.
What it gives: A large gear ratio in a small, light package. Almost no backlash: the teeth stay pressed together all the time.
What it costs: The same as in the lab with a large gear ratio: the actuator barely yields to a push and hits harder on impact.
Where it’s used: The joints of many robot manipulators, the rotary joints of Tesla Optimus.

Series elastic actuator

How it works: A spring of known stiffness sits between the gearbox and the link. A sensor measures how much it has compressed and derives the force: F = k · Δx. The controller controls the spring compression, that is, the force.
What it gives: Precise and stable force control. On impact the spring gives first and protects the gearbox.
What it costs: Force changes more slowly: the spring has to compress first. By the calculation of Wensing et al., the spring in a StarlETH leg joint lowers its natural frequency from about 80 to 13.5 Hz.
Where it’s used: Twelve actuators in ANYmal’s legs, the joints of the collaborative robot Baxter. The idea was proposed by Gill Pratt and Matthew Williamson at MIT in 1995.

Planetary roller screw

How it works: The motor turns a screw. Threaded rollers roll between the screw and the nut, and the nut travels along the screw. Through a rod, the nut turns the joint, like a muscle pulling on a bone.
What it gives: High linear force in a small package: the load is shared by many roller contact points. The Optimus linear actuator lifted a piano weighing about half a ton.
What it costs: The actuator is complex and expensive to make. The joint turns through a rod, so its angle is limited by the nut’s travel.
Where it’s used: Three types of linear actuators in Tesla Optimus.

Tendons

How it works: The motor sits in the body or the forearm, and the force reaches the joint through cables. That’s how forearm muscles move the fingers through tendons. A cable can only pull, so each joint gets a pair of cables or a cable with a spring.
What it gives: The limb is light and fast: there are no heavy motors in it. Many joints fit in a small volume, as in a hand.
What it costs: Cables stretch, rub against their guides and wear out; their tension has to be maintained.
Where it’s used: The low-inertia actuators of 1X NEO with 2% torque accuracy, many robotic hands.

Hydraulics

How it works: A pump pressurizes oil, a fast valve directs it into one chamber of the cylinder or the other, and the piston pushes the link. The pump sits in the body; only hoses run to the joints.
What it gives: Enormous force and power from a small actuator at the joint.
What it costs: The pump is noisy, oil can leak, and the system is heavy and hard to maintain.
Where it’s used: Excavators and other construction machinery. The old Atlas had 28 hydraulic joints; in 2024 Boston Dynamics switched to the electric Atlas.

There are other transmissions too. Cycloidal gearboxes sit in the joints of industrial robots: Nabtesco holds about 60% of that market. Digit 5 has its own cycloidal actuators in its legs.

Which actuator goes where

An actuator is chosen for the joint’s job. Sort eight parts of real robots by actuator type. For each one, think about what matters most to it: surviving impacts, high torque in a small package, pushing in a straight line, being light, or metering force precisely. You can drag a card, or click it and then click a bin.

No actuator beats the rest at everything: each wins at something and pays for it with something else. But any actuator only executes a command. To choose the next one, the policy needs to know how the last one turned out. Where does it learn that?

4 Sensors

What the robot knows about itself and the world

Did the joint turn to the right angle, where is the arm now, is the robot falling: the policy learns all of this only from sensors. And none of them can be trusted completely. Each has its weak spots: noise, drift, latency, rate.

Proprioception is sensing of the robot’s own body: joint encoders, motor current as a torque estimate, an inertial measurement unit (IMU) with a gyroscope and an accelerometer. Exteroception is sensing of the world: cameras, depth sensors, lidars, tactile sensors.

Below are the six sensors found most often in robots. The diagrams show the principle, not the actual construction. Orange marks what emits or presses, blue what the sensor measures, purple the elastic parts. Click a diagram to pause it.

Encoder

What it measures: Joint angle. Speed is computed from the difference of angles.
How it works: A slotted disk sits on the shaft. Light passes through a slot, and a sensor counts the pulses. A second sensor is offset by a quarter of a slot: whichever fires first tells the direction. Magnetic encoders use a magnet and a field sensor instead of light.
Weak spot: Speed from angle differences is noisy. The last bits of a cheap encoder are noise: the AS5047P in Mini Cheetah has 14 bits, about 2 of them noise.
Where it’s used: In every joint. Unitree G1 has two encoders per joint: on the motor rotor and at the gearbox output.

Inertial measurement unit (IMU)

What it measures: Acceleration and angular rate of the body about three axes.
How it works: In the accelerometer a tiny mass on springs shifts with tilt and acceleration; the shift is measured by capacitance. In the gyroscope a mass oscillates, and when the body rotates, the Coriolis force pushes it sideways. The g arrow in the diagram is the direction of gravity.
Weak spot: The gyroscope drifts if you integrate its readings over time. The accelerometer is noisy and confuses a jolt with a tilt. So the two are combined: that’s the lab below.
Where it’s used: In the body of every legged robot. Bosch BMI088 delivers gyroscope data up to 2,000 times per second.

Force-torque sensor

What it measures: Three forces and three torques: six numbers describing how an object pushes on the arm. In the diagram, Fx, Fy, Fz are forces along three axes, Mx, My, Mz are torques about them.
How it works: An elastic element sits between the wrist and the gripper. Under load it bends slightly, and strain gauges on it change resistance. The bending in the diagram is greatly exaggerated.
Weak spot: Expensive and fragile, drifts with temperature. Sees only the forces that pass through the wrist.
Where it’s used: In the UR5e flange. ATI Axia80 samples 8,000 times per second with under 1 ms latency. Franka FR3 has torque sensors in all seven joints.

Depth camera

What it measures: Distance to every point in the frame: a depth map.
How it works: Two cameras look from different positions. The closer an object, the more it shifts between their frames, and distance is computed from the shift. A projector paints dots on the scene so there is something to lock onto on smooth surfaces. The diagram is a top view.
Weak spot: Shiny and transparent objects, bright sunlight. The error grows with the square of distance: far away, the shift drops below a pixel.
Where it’s used: On humanoid heads and manipulator wrists. Intel RealSense D435i gives depth up to 1280×720 at 90 frames per second.

Lidar

What it measures: Distances around the robot: a point cloud.
How it works: A laser sends a short pulse and catches the reflection. Light covers a meter there and back in about 6.7 ns, and distance is computed from the time of flight. The beam sweeps the scene in a circle. The diagram is a top view of a room.
Weak spot: Few points: Livox MID-360 gives 200 thousand points per second, and the cloud updates 10 times per second. Glass and black surfaces barely reflect.
Where it’s used: On the heads of humanoids and quadrupeds, on warehouse robots and self-driving cars.

GelSight tactile sensor

What it measures: The shape and force of contact: the object’s imprint and the direction it is being pulled in.
How it works: A camera below films a transparent gel coated with a reflective film. The object presses the film in, side lighting casts shadows, and the relief is reconstructed from them. Markers in the gel shift under tangential force.
Weak spot: The gel wears out and tears, and the sensor is thicker than skin. There is still little ready-made data from such sensors.
Where it’s used: In the fingers of research hands. DIGIT (2020) weighs 20 g and captures 640×480 at 60 frames per second. Digit 360 (Meta, 2024) senses forces from 1 mN.

One sensor’s weak spots are covered by another. A simple example is body tilt, which the IMU measures. The gyroscope measures angular rate precisely, but if you integrate it, the error accumulates. The accelerometer sees the direction of gravity and is right on average, but very noisy. The scene shows three tilt estimates at once: gyroscope only, accelerometer only, and their blend. Next to the 10-second charts are the robot’s body and lines showing where each estimate thinks it is.

Gyro weight α0.5 Gyro zero offset1.5 °/s

Under the hood: the complementary filter

At each step we advance the estimate by the gyroscope and nudge it slightly toward the accelerometer angle:

\hat\theta_t = \alpha\,\bigl(\hat\theta_{t-1} + \omega_{\text{gyro}}\,\Delta t\bigr) + (1 - \alpha)\,\theta_{\text{accel}}

In frequency terms, it is a high-pass filter for the gyroscope and a low-pass filter for the accelerometer: fast changes come from the former, the slow "which way is down" from the latter. At $\alpha = 0.98$ and a 10 ms step, the time constant is about half a second. In the lab, the gyroscope has a zero offset of 1.5 °/s plus noise, and the accelerometer has noise with a 4° standard deviation plus spikes from body accelerations. The "Shake the robot" button adds four short jolts: the accelerometer takes them for tilt, while the gyroscope doesn’t see them, because it measures rotation. A Kalman filter does the same thing optimally when the noise models are known; that is Lesson 0.4.

On their own, the gyroscope drifts, and the accelerometer is noisy and confuses a jolt with a tilt. Together, in a complementary filter, they err by less than 2° even under shaking. But the robot’s parts run at very different rates: the IMU delivers data at up to 2 kHz, the camera at about 30 frames per second, the policy a few dozen times per second. How do they coexist in one system?

5 Rates

The rate ladder

A robot’s parts line up on a ladder of rates. These are the same levels from the start of the lesson: the lower the level, the faster it has to react. At the top, a slow model understands speech and decides what to do; at the bottom, the current loop drives the motor; and sensors sit at different levels. Sort ten components into frequency bands. After the check, the real ladder opens with examples from published systems.

The policy runs at levels from 10 to 200 Hz and sees only how the lower levels carried out its targets. It is often trained in a simulator. How accurately does the simulator reproduce what sits under the policy: the actuator with its controller?

🔬 Lab 2

An actuator model from data

The simulator reproduces the actuator imperfectly. A policy for a legged robot is trained in a simulator, where the actuator is ideal. A real one has latency, friction and torque saturation: the motor won’t give more than its limit, however hard you ask. And over time backlash appears: a gap in the gearbox where the motor is already turning while the link hasn’t moved yet. A policy tuned on an ideal actuator will get a different response on a real one.

You could try to derive an exact formula for the actuator. For a direct drive that works; for complex ones, such as ANYmal’s series elastic actuators, it doesn’t.

What remains is to learn the actuator model from data: record how the real actuator responds to commands and train a small network to reproduce it. We’ll find where the simulator and the actuator diverge, close the gap with a network, and check how much data that takes and what happens when the actuator wears out.

The top chart is the joint angle over 5 seconds. The dashed line is the command, that is, the angle the actuator’s controller should hold; the solid lines are the angles the real actuator and the models actually produced. The bottom chart shows by how many degrees each model diverges from the real actuator. The buttons under the chart change the command shape: a slow wave, a sum of two waves, a fast wave or sharp steps.

Latency15 ms Backlash0°

Training the network

Model error vs. amount of data

Under the hood: what’s inside

Real actuator. An internal PD position controller, 15 ms command latency, torque saturation, Coulomb and viscous friction. The link is a pendulum. Backlash is zero by default; the slider in the "Actuator wear" mission adds it. The encoder sits on the motor and doesn’t see the backlash: while the motor takes up the gap, no torque reaches the link.
Simulator model. The same PD, but instantaneous and without nonlinearities.
Data. 20 seconds of random commands at a 2.5 ms step: about 8 thousand samples. Input: position and velocity errors over the last 4 steps; output: the torque the actuator actually produced.
Network. 8 → 32 → 32 → 1 with tanh, Adam, 1,500 iterations of 64 samples regardless of data volume. After training, the network replaces the actuator model in the simulator. The network sees only the last 10 ms of history, so it learns latency longer than that poorly.

That is what Hwangbo et al. did in 2019: an exact analytical model can be built for a direct drive, but not for ANYmal’s series elastic actuators. They learned the actuator model from data, trained the policy in a simulator with that model, and the robot ran with no fine-tuning on hardware.

A network trained on 20 seconds of data reproduces the actuator 6 times more accurately than the ideal model, and 5 seconds is enough for an error under 1°. But the model knows only the actuator it was trained on: when the actuator wears out, it has to be retrained, and it learns long latency worse.

6 Check yourself

Eight common misconceptions

7 Takeaways

Lesson takeaways

A robot’s policy is an "observation → action" interface at a certain rate. It usually outputs targets for controllers, and a fast lower loop drives the motors.
Degrees of freedom are joints. Extra degrees of freedom give freedom of pose for the same gripper position.
The gear ratio trades speed for torque, but inertia grows as $N^2$ : at large $N$ the actuator loses backdrivability and becomes more dangerous on impact.
An actuator is chosen for the joint’s job: for impacts and contact, a small gear ratio or a spring; for high torque in a small package, a harmonic drive; for linear force, a roller screw; for a light limb, tendons.
Sensors are noisy, they drift and they lag. They are fused: from the complementary filter to the Kalman filter.
The gap between simulator and robot often starts at the actuator. It is narrowed with an actuator model learned from data.

Open questions

Why do a robot vacuum and a quadruped robot have such different actuators?

The vacuum only needs wheel speed, and there are almost no impact contacts, so cheap motors with a large gearbox will do. A quadruped hits the ground on every step and has to feel the contact force, so it needs transparent actuators with a small gear ratio.

You are training a policy for a hand with 20 degrees of freedom. Which sensors would you add to the cameras, and why?

Encoders on every joint for precise finger positions, torque estimates from current or torque sensors to feel the grip force, and tactile sensors on the fingertips: the camera can’t see contact when the fingers cover the object.

Paper guide

Wensing et al., "Proprioceptive Actuator Design in the MIT Cheetah", IEEE T-RO, 2017.

Read closelyWhy a proprioceptive actuator is needed, how an actuator with a large motor gap radius is built, and how force is estimated from current.
Can skipThe details of the motor’s electromagnetic design.
Question to keep in mindWhy does a series spring lower the leg’s natural frequency, and what does that mean for control?

Resources

Learning agile and dynamic motor skills for legged robotsJ. Hwangbo et al., Science Robotics 2019 — an actuator model from data
Mini Cheetah: backflipMIT video — quasi-direct drives in action
Farewell to HD AtlasBoston Dynamics says goodbye to the hydraulic Atlas

➡️

What’s next: kinematics and control. We have covered what the policy controls. The remaining question is how its target becomes motion. In the arm lab, the joint angles were found on their own as soon as you dragged the gripper, and the joint controllers simply held the target. Who converts a gripper pose into joint angles, how does a controller turn a target into torque, and why can a neural network that has learned kinematics miss by more than 40 cm? Lesson 0.3.

Was the lesson clear?

Found a mistake or inaccuracy? Report it on GitHub

Found this lesson useful? Say thanks — that is how the author knows the course is being read.