Screw Sorting
This example demonstrates how to use Tzara, the Telekinesis Physical AI Agent, to build a screw sorting pipeline. Tzara generates code that detects screws in a blue bin using Grounding DINO, rejects oversized bounding boxes, segments each screw with SAM, and classifies it as black, large gray, or small gray based on mask color and area. The gripper picks each screw at a hardcoded height with a PCA-derived orientation and places it in one of three boxes. Full Rerun visualization and detailed logging are included at every stage.
Demo
UR10e sorting three types of screws by color and size into separate boxes
The Natural Language Instruction
In the blue bin there are 3 types of screws, one is black two are gray. One gray type is bigger than the other gray type. Can you sort them in 3 boxes. Differentiate the two gray ones by mask size. Reject big bboxes returned by dino or qwen. Hardcode grasp height to 0.01. Do PCA on the mask for grasping orientation. 3 boxes poses [0.3041, 0.85220, 0.086, 180.0, 0.0, 180.0], [0.3041, 0.73786, 0.086, 180.0, 0.0, 180.0], [0.3041, 0.62573, 0.086, 180.0, 0.0, 180.0]. Add detailed logging and Rerun visualization for every stage and step so we can see what is going on. The goal is robust debugging and traceability so failures can be diagnosed easily.About this demonstration
This demonstration used an earlier SDK version. Breaking API changes have made the original code incompatible with the current SDK, so we have temporarily removed it. Updated code is coming soon.
Try It Yourself
Set up Tzara, open it in your configured workspace, and use the instructions above as a starting prompt. Review the generated code before running it.
For current APIs and examples, see Robot Motion.