π0.5: Vision-Language-Action Model
SUMMARY
π0.5 is a large, pretrained Vision-Language-Action policy from Physical Intelligence: a vision-language backbone paired with a flow-matching action head, pretrained across many robots, tasks, and environments before it ever sees your task.
Instead of training a policy from scratch, you fine-tune the pretrained checkpoint on your own demonstrations. The cross-embodiment pretraining is what lets it follow a free-form language instruction and hold up in environments it wasn't shown at fine-tuning time.
When to Use π0.5?
Reach for π0.5 when:
- The deployment environment or object set will vary, and the policy needs to generalize rather than memorize one scene
- Instructions should be given as language ("pick up the blue mug") rather than baked into a fixed task ID
- Demonstrations span multiple tasks or embodiments, so pretrained cross-task knowledge is worth more than a policy trained from zero
- A GPU capable of hosting a large vision-language backbone is available for fine-tuning and inference
For a single-task policy that trains fast on a handful of teleoperated demonstrations and needs no pretrained checkpoint, see ACT instead. For a pretrained policy that leans on dynamics-aware world modeling rather than language conditioning, see Fast-WAM.
Requirements
| Requirement | Value |
|---|---|
| Python | 3.11-3.12 |
| GPU | A CUDA GPU with enough memory to fine-tune a vision-language backbone (see the checkpoint's card for the exact figure) |
| Pretrained checkpoint | Downloaded once and reused across fine-tuning runs |
| Demonstrations | A dataset with camera frames, proprioceptive state, the action taken at each step, and a language instruction per episode. Currently, only the LeRobot dataset format is supported |
Quick Installation
pip install "telekinesis-cerebellum[pi05]"The pi05 extra brings the π0.5 model and its flow-matching training loop. Everything else - dataset loading, checkpointing, and the deployment interface shared with ACT - comes with the base package.
Quickstart
Download the pretrained checkpoint
cerebellum download pi05-base --to checkpoints/pi05-baseLoad your demonstration data from LeRobot
Each episode needs a language instruction alongside its camera frames, proprioceptive state, and actions - LeRobot datasets carry a per-episode task string already, so it comes along with the rest:
from telekinesis.dataengine import load_lerobot_dataset
# Load a LeRobot dataset of teleoperated episodes, with language instructions
dataset = load_lerobot_dataset(
repo_id="telekinesis-ai/tabletop_pick_and_place", # local path or LeRobot Hub repo id
)
print(dataset[0])Create config.yaml
task: tabletop_pick_and_place
dataset:
path: demos/
cameras: [cam_high, cam_wrist]
policy:
class_name: Pi05
pretrained_checkpoint: checkpoints/pi05-base
action_horizon: 50 # actions predicted per flow-matching step
training:
num_steps: 30000
batch_size: 32
learning_rate: 0.00002
freeze_backbone: false # true for a lighter, faster fine-tune
device: auto
logger:
log_dir: logs
experiment: tabletop_pi05
save_interval: 1000Deploy
from telekinesis.cerebellum.policy import Policy
from teklkines
# Use a synapse robot
robot =
policy = Policy("logs/tabletop_pi05/checkpoint_best.pt")
observation = robot.get_observation() # camera frames + proprioceptive state
action_chunk = policy.get_action(observation, instruction="pick up the blue mug")
robot.execute(action_chunk)Tuning Notes
- Freezing the backbone fine-tunes only the action head - faster and less prone to overfitting on a small dataset, at the cost of adapting less to visuals that differ sharply from pretraining.
- Action horizon is the flow-matching equivalent of ACT's chunk size - the same reactivity-versus-smoothness trade-off applies.
- Mixing in a handful of demonstrations from related tasks, when available, tends to help more than adding more of the same task - that is the cross-task transfer the pretraining is there to unlock.