Telekinesis Agentic Skill Library

Appearance

Illusion - Generating a Dataset with a Worker ​

Goal: Generate, merge, and split a complete bin-picking dataset from a spec YAML.

Level: Intermediate

Time: ~20 minutes

Background ​

The quickstarts build a scene in Python, which is the right way to experiment. A production dataset is different: it runs for hours, has to survive interruptions, and has to end up in a format a trainer can read.

A Worker covers that. The whole dataset – models, categories, placement, camera, physics, output format – lives in one spec YAML. The worker generates the images in shards, merges them into a single annotation file, and splits them into train, valid, and test.

In this tutorial you will run the bundled bin-picking spec, then change it to produce your own dataset.

1. Run the Bundled Spec ​

From the telekinesis-illusion repository root:

bash
python examples/generate_synthetic_data_with_bin_picking_worker.py \
  --spec-file example_bin_picking_gearwheel_2.yaml

This generates 20 scenes of gearwheels in an industrial bin, with pipes and pipe fixtures as labeled distractors, then merges the shards and opens the result in FiftyOne.

Bare filenames resolve against the configs/ directory. Pass --no-preview to skip the viewer.

INFO

For a first run, lower metadata.num_images and shard.size in the spec – both to 10, for example – so a full generate and merge cycle finishes in minutes rather than hours.

2. Read the Output Layout ​

The run writes shards under the dataset root, then a merged annotation file across them:

output/
└── example_bin_picking_gearwheel_2/
    ├── merged_coco_annotations.json
    ├── shard_20260903_101500_a1b2c3d4/
    │   ├── coco_annotations.json
    │   └── images/
    │       ├── 000000.png
    │       └── ...
    └── shard_20260903_104512_e5f6a7b8/
        ├── coco_annotations.json
        └── images/

Each shard is a self-contained mini-COCO dataset, so a run that stops early still leaves usable data behind. Train against merged_coco_annotations.json, whose image paths resolve relative to the dataset root – not against the individual shards.

Because the bundled spec sets output.dataset_format: coco, merging also writes a split dataset with train/, valid/, and test/ directories, each with its own _annotations.coco.json.

3. Point the Spec at Your Own Part ​

Copy the bundled spec and change the target model. Every path in models[].path is relative to metadata.asset_directory:

yaml
metadata:
  asset_directory: ../assets
  dataset_name: my_bracket_dataset
  num_images: 200
  base_output_directory: output

shard:
  size: 50

min_number_visible_models: 1
max_number_visible_models: 6
models:
  - name: bracket_01
    id: 1
    supercategory: part
    category_name: bracket
    path: models/mechanical_parts/bracket_01.glb
    instances: { min: 0, max: 6 }
    simulation: { active: true, collision_shape: CONVEX_HULL }
    scale: 1.0
    preprocess_model: True

  - name: plastic_bin_2
    id: None
    supercategory: container
    category_name: distractor
    path: models/bins/plastic_bin_2.glb
    instances: { min: 0, max: 1 }
    simulation: { active: false, collision_shape: MESH }
    scale: 1.0
    preprocess_model: True

The supercategory is what routes an asset through the randomizers: part objects are placed in the container, container objects are picked one per scene, and distractor objects use their own visibility bounds.

Setting id: None with category_name: distractor keeps the bin out of the annotations. Give a distractor a real id and category_name instead if you want the model to learn it as a class.

4. Choose the Output Format ​

Add the split configuration so the dataset comes out training-ready:

yaml
output:
  shard_name_template: "shard_{date}_{uuid}"
  max_size_gb: 50
  dataset_format: yolo      # None, coco, or yolo
  train_val_tes_ratio: [0.7, 0.2, 0.1]
  stratify: True
  seed: 42

With dataset_format: yolo, merging also writes an Ultralytics YOLO-seg dataset with a data.yaml. Use coco for RF-DETR and other COCO trainers, or None to keep only the merged COCO file and split it later with DatasetConverter.

max_size_gb is a safety cap: generation stops after the first shard that pushes the dataset past it.

5. Run and Inspect ​

bash
python examples/generate_synthetic_data_with_bin_picking_worker.py \
  --spec-file my_bracket_spec.yaml

To inspect the result later, set DATASET_DIR at the top of examples/view_dataset.py to the dataset root and run:

bash
python examples/view_dataset.py

The script detects the layout, prints a per-class distribution table across the splits, and opens the FiftyOne app. Check the table before training: a class that appears in the spec but barely appears in the table usually means its instance bounds or the grid capacity are too tight.

Summary ​

You have:

  • Run a complete, sharded dataset from a spec YAML.
  • Read the shard and merged output layout, and understood which file to train on.
  • Retargeted the spec to a different part and chosen the output dataset format.

Next Steps ​