Workers

A worker turns a single spec YAML into a complete dataset. It builds the Context and the randomizer tree for a use case, generates the images in shards, and merges those shards into one annotation file – optionally split into train, valid, and test.

Use a worker when you want a dataset. Use the Python API directly when you are experimenting with a scene.

Import

from telekinesis.illusion.workers.bin_picking_worker import BinPickingWorker

Running a Worker

worker = BinPickingWorker(spec_file_path)
worker.generate()
worker.merge_shards(preview=True)

Method	Description
generate()	Generate every shard defined by the spec.
merge_shards(preview=True)	Merge the shards into merged_coco_annotations.json, run the dataset split when the spec requests one, and optionally open the result in the viewer.

The bundled entry script does the same from the command line:

python examples/generate_synthetic_data_with_bin_picking_worker.py \
  --spec-file example_bin_picking_gearwheel_2.yaml

Bare filenames resolve against the repository's configs/ directory; an absolute or relative path points at a spec elsewhere. Pass --no-preview to skip opening the merged dataset afterwards.

Smoke test first

Lower metadata.num_images and shard.size (for example both to 10) before a long run, so a full generate and merge cycle finishes in minutes.

Shards

A shard is a self-contained mini-COCO dataset: one coco_annotations.json plus an images/ directory of rendered PNGs. Shards let a run checkpoint its progress, so a size limit or an interruption never discards finished work.

• num_shards = ceil(metadata.num_images / shard.size). The total number of scenes across all shards always equals metadata.num_images exactly.
• shard.size counts scenes, not images. Each scene renders camera_pose_randomizer.number_of_views images.
• Generation stops after the first shard whose total dataset size reaches output.max_size_gb.
• Shards must be merged before use. Consumers load merged_coco_annotations.json, whose image paths resolve relative to the dataset root.

<base_output_directory>/
└── <dataset_name>/
    ├── merged_coco_annotations.json     # written by merge_shards()
    ├── shard_<date>_<uuid>/
    │   ├── coco_annotations.json
    │   └── images/
    │       ├── 000000.png
    │       └── ...
    └── shard_<date>_<uuid>/
        ├── coco_annotations.json
        └── images/

BinPickingWorker

BinPickingWorker generates parts-in-bin scenes: target parts and distractors are placed on the upper face of a container, settled with physics, and rendered against randomized backgrounds.

Given a spec, it registers every entry in models: and distractors: on the context, then wires the randomizers in a fixed order: object instances, container instance (always exactly one), object poses on the visible container, distractor instances, distractor poses, materials per supercategory, background, and camera pose. Per shard it runs the generation loop, then checks the dataset size against the configured limit.

Spec YAML

A spec is a self-contained dataset definition. The reference spec is configs/example_bin_picking_gearwheel_2.yaml.

metadata:
  asset_directory: ../assets
  dataset_name: example_bin_picking_gearwheel_2
  use_case: instance_segmentation
  annotation_format: coco_instances_rle
  num_images: 20
  base_output_directory: output

shard:
  size: 10

min_number_visible_models: 1
max_number_visible_models: 6
models:
  - name: gearwheel_2
    id: 1
    supercategory: part
    category_name: gear
    path: models/mechanical_parts/gearwheel_2.glb
    instances: { min: 0, max: 6 }
    simulation: { active: true, collision_shape: CONVEX_HULL }
    scale: 1.0
    preprocess_model: True

  - name: plastic_bin_2
    id: None
    supercategory: container
    category_name: distractor
    path: models/bins/plastic_bin_2.glb
    instances: { min: 0, max: 1 }
    simulation: { active: false, collision_shape: MESH }
    scale: 1.0
    preprocess_model: True

Metadata and Sharding

Key	Type	Purpose
metadata.dataset_name	string	Dataset directory name under the output root.
metadata.num_images	int	Total scenes generated across all shards.
metadata.base_output_directory	string | null	Output root. Null or empty falls back to ./output/<dataset_name> relative to the run directory.
metadata.asset_directory	string | null	Directory holding models/, hdris/, and materials/. Relative paths are anchored on the spec file.
metadata.use_case	string	Recorded in the manifest only.
metadata.annotation_format	string	Recorded in the manifest only.
metadata.package_version	string	Recorded in the manifest only.
metadata.info / metadata.licenses	map / list	COCO info and licenses blocks copied into the annotations.
shard.size	int	Scenes per shard.

Models and Distractors

Key	Type	Purpose
models[].name	string	Unique asset name.
models[].id	int | None	COCO category_id. Several entries may share an id to map multiple meshes to one category. None excludes the asset from the annotations.
models[].supercategory	part | container | distractor	Drives randomizer routing.
models[].category_name	string	COCO category name. Must be consistent across entries sharing an id.
models[].path	string	Model path relative to metadata.asset_directory.
models[].instances.min / .max	int	Per-scene instance count range for this asset.
models[].simulation.active	bool	Whether the asset participates in physics. Containers are typically false.
models[].simulation.collision_shape	CONVEX_HULL | MESH	Collider type. Use MESH for bins.
models[].scale	float	Uniform scale.
models[].preprocess_model	bool	Run import-time preprocessing.
min_number_visible_models / max_number_visible_models	int	Bounds on visible target objects per scene.
min_number_visible_distractors / max_number_visible_distractors	int	Bounds on visible distractors per scene.
distractors[]	list	Same shape as models[]. Omit the list to disable distractors.

To label distractors, give them a real id and category_name. To keep them unlabeled, set id: None and category_name: distractor.

Placement

Key	Type	Purpose
pose_sampling.strategy	random | grid	Placement strategy for target objects on the container's upper face.
pose_sampling.params.min_height / max_height	float	Vertical band above the container's upper face in which object centres are sampled.
pose_sampling.params.face_sample_range	[float, float]	Fractional inset on the container's upper face. Default [0.25, 0.75].
pose_sampling.params.grid.rows / .cols / .layers	int	Grid dimensions, used when strategy: grid. Grid capacity caps the number of visible parts.
pose_sampling.params.grid.layer_spacing	float	Vertical spacing between grid layers.
pose_sampling.params.grid.xy_jitter	float	Random XY offset applied per grid cell.
pose_sampling.params.grid.z_rotation_range	[float, float]	Bounds on per-instance Z rotation, in degrees.
pose_sampling.params.grid.shuffle	bool	Whether grid cell assignment is shuffled.

Camera and Renderer

Key	Type	Purpose
camera.field_of_view	float	Field of view, in radians.
camera.clip_start / clip_end	float	Near and far clip planes.
camera.image_width / image_height	int	Output resolution.
camera.pixel_aspect_x / pixel_aspect_y	float	Pixel aspect ratio.
camera.shift_x / shift_y	float	Lens shift.
renderer.image_format	string	Rendered image format, for example PNG.
camera_pose_randomizer.sampler	string	Camera pose sampler, for example volume_sampler.
camera_pose_randomizer.number_of_views	int	Rendered images per scene.
camera_pose_randomizer.params.distance_range	[float, float]	Bounds on the camera distance from the point of interest.
camera_pose_randomizer.params.inplane_rot_min / inplane_rot_max	float	Bounds on in-plane camera rotation, in degrees.
camera_pose_randomizer.params.point_of_interst	[float, float, float]	World-space point the camera looks at. Spelled as shown; copy the key verbatim.

Randomization Overrides

Key	Type	Purpose
instance_randomizer.<role>.min / .max	int	Per-supercategory override of the visible-instance bounds. Roles are part, container, and distractor.
material_randomizer.<role>	list[string]	Material type tags per supercategory. Defaults: part: [metal], container: [plastic], distractor: [metal]. An empty list disables material randomization for that role.
background_randomizer.categories	list[string]	HDRI categories the background is sampled from.

Physics

Key	Type	Purpose
physics_simulator.active	bool	Whether to settle the objects before rendering each scene.
physics_simulator.min_simulation_time_range / max_simulation_time_range	[float, float]	Simulated-seconds ranges sampled per scene.
physics_simulator.check_object_interval	float	Interval, in simulated seconds, between at-rest checks.
physics_simulator.object_stopped_location_threshold	float	Maximum location change, in meters, for an object to count as at rest.
physics_simulator.object_stopped_rotation_threshold	float	Maximum rotation change, in radians, for an object to count as at rest.
physics_simulator.substeps_per_frame	int	Physics substeps per simulation frame.
physics_simulator.solver_iters	int	Solver iterations.
physics_simulator.use_volume_com	bool	Compute the center of mass from volume instead of mesh vertices.
physics_simulator.verbose	bool	Verbose simulation logging.

Output

Key	Type	Purpose
output.shard_name_template	string	Shard directory name. Supports {date} (YYYYMMDD_HHMMSS) and {uuid} (8 hex characters). Default shard_{date}_{uuid}.
output.write_manifest	bool	Whether to write a manifest.
output.max_size_gb	number	Early-stop threshold for the total dataset size, in GiB. Default 10.
output.dataset_format	coco | yolo | null	When set, merge_shards() also runs a DatasetConverter split into this format.
output.train_val_tes_ratio	[float, float, float]	Train, valid, and test ratios for that split. Default [0.7, 0.2, 0.1]. Spelled as shown; copy the key verbatim.
output.stratify	bool	Whether the split is stratified by dominant category. Default true.
output.seed	int	RNG seed for the split. Default 42.

Common Modifications

Goal	Change
Add a target part	Append an entry to models: with supercategory: part, a unique name, a model path, and an id plus category_name.
Map several meshes to one class	Give every entry the same id and category_name.
Swap the bin	Add or replace an entry with supercategory: container, simulation.active: false, and collision_shape: MESH. Exactly one container is picked per scene.
Add clutter	Raise max_number_visible_models and max_number_visible_distractors, and raise the per-asset instances.max so the instance pool is large enough.
Resize the dataset	Change metadata.num_images and shard.size, using output.max_size_gb as a safety cap.
Change resolution	Edit camera.image_width, camera.image_height, and camera.field_of_view (radians).

Dataset Utilities

Merge shards, split into train/valid/test, convert to COCO or YOLO, and inspect the result in FiftyOne.

Read more →