Telekinesis Skill Examples
Prerequisites
New here? Complete the Quickstart first — it sets up your API key, conda environment, and SDK in under 5 minutes.
Run an Example!
Run any example from telekinesis-examples repository
Clone the repository and the telekinesis-data submodule
git clone --depth 1 --recurse-submodules --shallow-submodules \
https://github.com/telekinesis-ai/telekinesis-examples.git
cd telekinesis-examplesWhat gets downloaded?
The flags above keep the clone small:
--depth 1fetches only the latest commit, not full history.--recurse-submodules --shallow-submodulesalso pulls thetelekinesis-datasubmodule (sample images, point clouds, robot states), again atdepth 1. Roughly 1.5 GB of free disk space is required for the telekinesis-data.
When you start using Telekinesis on your own data, you can ignore the telekinesis-data submodule - it exists purely to make the examples runnable out of the box.
Install dependencies for examples
conda activate telekinesis
pip install rerun-sdk==0.31.2 pycocotools scipyRun an example (see more examples below)
Runs segment_image_using_sam and visualizes masks in Rerun.
python examples/cornea_examples.py --example segment_image_using_sam
SAM masks predicted from the sample input image.
Cornea
List all available examples and run one:
python examples/cornea_examples.py --listpython examples/cornea_examples.py --example <name>| Example name | What it covers |
|---|---|
segment_image_using_rgb | Segments regions by thresholding raw RGB color channels. |
segment_image_using_hsv | Segments regions by thresholding hue, saturation, and value. |
segment_image_using_lab | Segments regions using perceptually uniform LAB color space. |
segment_image_using_ycrcb | Segments regions by separating luminance from chroma components. |
segment_image_using_focus_region | Isolates a rectangular region of interest for downstream processing. |
segment_image_using_watershed | Segments touching or overlapping objects using the watershed algorithm. |
segment_image_using_flood_fill | Segments a contiguous region by flood-filling from a seed point. |
segment_image_using_foreground_birefnet | Extracts foreground from background using the BiRefNet deep learning model. |
segment_image_using_sam | Segments arbitrary objects using Meta's Segment Anything Model (SAM). |
segment_image_using_grab_cut | Separates foreground from background using the GrabCut graph-cut algorithm. |
segment_image_using_felzenszwalb | Over-segments an image into superpixels using Felzenszwalb's method. |
segment_image_using_slic_superpixel | Groups pixels into compact superpixels using the SLIC algorithm. |
filter_segments_by_area | Removes segments outside a specified minimum or maximum area. |
filter_segments_by_color | Keeps only segments whose average color falls within a target range. |
filter_segments_by_mask | Filters segments based on overlap with a provided binary mask. |
segment_image_using_otsu_threshold | Binarizes an image by automatically finding the optimal Otsu threshold. |
segment_image_using_local_threshold | Applies adaptive thresholding over local image neighborhoods. |
segment_image_using_yen_threshold | Binarizes an image using Yen's maximum correlation thresholding method. |
segment_image_using_threshold | Segments an image using a fixed manually specified threshold value. |
segment_image_using_adaptive_threshold | Computes per-pixel thresholds based on local mean or Gaussian-weighted neighborhood. |
segment_image_using_laplacian_threshold | Detects and segments edges by thresholding the Laplacian of the image. |
python examples/cornea_examples.py --example segment_image_using_rgbRetina
List all available examples and run one:
python examples/retina_examples.py --listpython examples/retina_examples.py --example <name>| Example name | What it covers |
|---|---|
detect_circle_using_classic_hough | Detects circles in an image using the classical Hough Circle Transform. |
detect_contours | Finds and draws object contours using edge-based detection. |
detect_objects_using_yolox | Detects objects using the YOLOX neural network model. |
detect_objects_using_rfdetr | Detects objects using the RF-DETR transformer-based detection model. |
detect_objects_using_qwen | Detects and describes objects using the Qwen vision-language model. |
detect_objects_using_grounding_dino | Detects objects from free-text prompts using Grounding DINO. |
python examples/retina_examples.py --example detect_contoursPupil
List all available examples and run one:
python examples/pupil_examples.py --listpython examples/pupil_examples.py --example <name>| Example name | What it covers |
|---|---|
filter_image_using_morphological_erode | Shrinks bright regions by eroding pixel boundaries. |
filter_image_using_morphological_dilate | Expands bright regions by dilating pixel boundaries. |
filter_image_using_morphological_close | Fills small holes and gaps by closing morphological regions. |
filter_image_using_morphological_open | Removes small noise while preserving larger structures. |
filter_image_using_morphological_gradient | Highlights edges by computing the morphological gradient. |
filter_image_using_morphological_tophat | Extracts small bright features against a dark background. |
filter_image_using_morphological_blackhat | Extracts small dark features against a bright background. |
filter_image_using_morphological_hitmiss | Detects specific pixel patterns using hit-or-miss morphology. |
filter_image_using_frangi | Enhances tubular and vessel-like structures using the Frangi filter. |
filter_image_using_hessian | Highlights blob and ridge structures using Hessian-based filtering. |
filter_image_using_sato | Detects tubular structures using the Sato vesselness filter. |
filter_image_using_meijering | Enhances neurite-like ridges using the Meijering filter. |
filter_image_using_laplacian | Sharpens edges by applying the Laplacian second-derivative filter. |
filter_image_using_sobel | Detects edges by computing the Sobel gradient magnitude. |
filter_image_using_scharr | Detects edges with improved rotational accuracy using the Scharr operator. |
filter_image_using_gabor | Extracts texture features at specific orientations and frequencies using a Gabor filter. |
filter_image_using_bilateral | Smooths an image while preserving sharp edges using bilateral filtering. |
filter_image_using_box | Applies a uniform box blur to smooth the image. |
filter_image_using_gaussian_blur | Smooths an image using a Gaussian kernel to reduce noise. |
filter_image_using_median_blur | Removes salt-and-pepper noise using median filtering. |
filter_image_using_blur | Applies a simple average blur to the image. |
enhance_image_using_auto_gamma_correction | Automatically adjusts image brightness using gamma correction. |
enhance_image_using_white_balance | Corrects color casts by adjusting white balance. |
enhance_image_using_clahe | Improves local contrast using Contrast Limited Adaptive Histogram Equalization. |
transform_image_using_pyramid_downsampling | Reduces image resolution using Gaussian pyramid downsampling. |
transform_image_using_pyramid_upsampling | Increases image resolution using Laplacian pyramid upsampling. |
transform_mask_using_blob_thinning | Skeletonizes binary mask blobs to single-pixel-wide lines. |
calculate_image_pca | Computes principal components of pixel data for dimensionality analysis. |
calculate_image_centroid | Calculates the centroid of intensity or binary regions in an image. |
convert_image_color_space | Converts an image between color spaces (e.g. BGR to HSV). |
normalize_image_intensity | Rescales pixel intensities to a standard range. |
split_image_into_channels | Splits a multi-channel image into individual channel images. |
merge_image_from_channels | Merges separate channel images into a single multi-channel image. |
resize_image | Resizes an image to a specified width and height. |
resize_image_with_aspect_fit | Resizes an image while preserving its aspect ratio. |
rotate_image | Rotates an image by a given angle around its center. |
translate_image | Shifts an image by a specified pixel offset. |
pad_image | Adds padding around the image borders. |
crop_image_center | Crops a region from the center of an image. |
crop_image_using_bounding_boxes | Crops regions from an image using bounding box coordinates. |
crop_image_using_polygon | Crops and masks a region defined by a polygon. |
bitwise_and_images | Combines two images using a bitwise AND operation. |
bitwise_or_images | Combines two images using a bitwise OR operation. |
bitwise_xor_images | Highlights differences between two images using bitwise XOR. |
bitwise_difference_images | Computes the absolute pixel difference between two images. |
bitwise_not_image | Inverts pixel values using a bitwise NOT operation. |
overlay_images_using_weighted_overlay | Blends two images together using configurable weights. |
project_pixel_to_camera_point | Back-projects a 2D pixel to a 3D camera-frame point using depth. |
project_camera_point_to_pixel | Projects a 3D camera-frame point onto the 2D image plane. |
project_pixel_to_world_point | Back-projects a 2D pixel to a 3D world-frame point using depth and pose. |
project_world_point_to_pixel | Projects a 3D world-frame point onto the 2D image plane. |
python examples/pupil_examples.py --example filter_image_using_sobelVitreous
List all available examples and run one:
python examples/vitreous_examples.py --listpython examples/vitreous_examples.py --example <name>| Example name | What it covers |
|---|---|
calculate_axis_aligned_bounding_box | Computes the axis-aligned bounding box of a point cloud. |
calculate_oriented_bounding_box | Computes the tightest oriented bounding box around a point cloud. |
calculate_point_cloud_centroid | Calculates the geometric centroid of a point cloud. |
calculate_points_in_point_cloud | Returns the total number of points in a point cloud. |
calculate_plane_normal | Estimates the normal vector of a planar surface in a point cloud. |
cluster_point_cloud_using_dbscan | Groups points into clusters using the DBSCAN density-based algorithm. |
cluster_point_cloud_based_on_density_jump | Segments clusters by detecting jumps in local point density. |
convert_mesh_to_point_cloud | Samples a point cloud from the surface of a 3D mesh. |
create_cylinder_mesh | Creates a parametric cylinder mesh. |
create_plane_mesh | Creates a parametric plane mesh. |
create_sphere_mesh | Creates a parametric sphere mesh. |
create_torus_mesh | Creates a parametric torus mesh. |
estimate_principal_axis_within_radius | Estimates the principal axis of points within a specified radius. |
estimate_principal_axes | Computes the principal axes of an entire point cloud using PCA. |
filter_point_cloud_using_pass_through_filter | Removes points outside a specified range along one axis. |
filter_point_cloud_using_bounding_box | Keeps only points within an axis-aligned bounding box. |
filter_point_cloud_using_cylinder_base_removal | Removes points belonging to a detected cylinder base. |
filter_point_cloud_using_mask | Filters points using a boolean mask array. |
filter_point_cloud_using_oriented_bounding_box | Keeps only points within an oriented bounding box. |
filter_point_cloud_using_plane_proximity | Retains points within a distance threshold from a fitted plane. |
filter_point_cloud_using_plane_defined_by_point_normal_proximity | Retains points near a plane defined by a point and normal vector. |
filter_point_cloud_using_plane_splitting | Splits a point cloud into two halves along a plane. |
filter_point_cloud_using_radius_outlier_removal | Removes points with too few neighbors within a given radius. |
filter_point_cloud_using_statistical_outlier_removal | Removes statistical outliers based on mean distance to neighbors. |
filter_point_cloud_using_uniform_downsampling | Reduces point count by keeping every nth point uniformly. |
filter_point_cloud_using_viewpoint_visibility | Removes points not visible from a specified camera viewpoint. |
filter_point_cloud_using_voxel_downsampling | Downsamples a point cloud by averaging points within voxel cells. |
add_point_clouds | Merges two point clouds into one. |
subtract_point_clouds | Removes points from one cloud that overlap with another. |
scale_point_cloud | Scales a point cloud by a given factor. |
apply_transform_to_point_cloud | Applies a rigid 4×4 transformation matrix to a point cloud. |
project_point_cloud_to_plane | Projects all points onto a fitted plane. |
project_point_cloud_to_plane_defined_by_point_normal | Projects all points onto a plane defined by a point and normal. |
reconstruct_mesh_using_convex_hull | Reconstructs a surface mesh as the convex hull of a point cloud. |
reconstruct_mesh_using_poisson | Reconstructs a watertight surface mesh using Poisson reconstruction. |
register_point_clouds_using_cuboid_translation_sampler_icp | Aligns point clouds using ICP with a cuboid translation sampler initialization. |
register_point_clouds_using_fast_global_registration | Aligns point clouds using Fast Global Registration for coarse alignment. |
register_point_clouds_using_point_to_plane_icp | Refines point cloud alignment using point-to-plane ICP. |
register_point_clouds_using_point_to_point_icp | Refines point cloud alignment using point-to-point ICP. |
register_point_clouds_using_rotation_sampler_icp | Aligns point clouds using ICP with a rotation sampler initialization. |
segment_point_cloud_using_color | Segments a point cloud into regions by color similarity. |
segment_point_cloud_using_plane | Segments the dominant plane from a point cloud using RANSAC. |
python examples/vitreous_examples.py --example scale_point_cloudDatatypes
List all available examples and run one:
python examples/datatypes_examples.py --listpython examples/datatypes_examples.py --example <name>| Example name | What it covers |
|---|---|
bool | Demonstrates the Telekinesis boolean datatype. |
float | Demonstrates the Telekinesis float datatype. |
int | Demonstrates the Telekinesis integer datatype. |
float32 | Demonstrates the 32-bit float datatype. |
string | Demonstrates the Telekinesis string datatype. |
rgba32 | Demonstrates the RGBA32 color datatype. |
image | Demonstrates the Telekinesis image datatype. |
imageformat | Demonstrates supported image format encodings. |
mat3x3 | Demonstrates the 3×3 matrix datatype. |
mat4x4 | Demonstrates the 4×4 transformation matrix datatype. |
vector3d | Demonstrates the 3D vector datatype. |
vector4d | Demonstrates the 4D vector datatype. |
points2d | Demonstrates the 2D point array datatype. |
points3d | Demonstrates the 3D point array datatype. |
points3d_list | Demonstrates a list of 3D point arrays. |
boxes2d | Demonstrates the 2D bounding box datatype. |
boxes3d | Demonstrates the 3D bounding box datatype. |
mesh3d | Demonstrates the 3D mesh datatype. |
linestrips2d | Demonstrates the 2D line strip datatype. |
circles | Demonstrates the circle geometry datatype. |
ellipses | Demonstrates the ellipse geometry datatype. |
lines | Demonstrates the line geometry datatype. |
channeldatatype | Demonstrates the channel data type descriptor. |
colormodel | Demonstrates the color model datatype. |
categories | Demonstrates the category label datatype. |
object_detection_annotations | Demonstrates bounding-box object detection annotation format. |
panoptic_segmentation_annotation | Demonstrates panoptic segmentation annotation format. |
geometry_detection_annotations | Demonstrates geometric shape detection annotation format. |
part | Demonstrates the part datatype for object decomposition. |
tray | Demonstrates the tray datatype for bin-picking scenes. |
bin | Demonstrates the bin datatype for bin-picking scenes. |
python examples/datatypes_examples.py --example boolMedulla
Each script is run directly with:
python examples/medulla/<vendor>/<script>.pyWebcam
| Script | What it covers |
|---|---|
quickstart.py | Minimal example to capture a frame from a webcam. |
capture_image_example.py | Captures a single image from a webcam. |
capture_video_example.py | Captures a video stream from a webcam. |
camera_loop.py | Continuously reads frames from a webcam in a loop. |
publish_video_example.py | Publishes a webcam video stream over BabyROS. |
python examples/medulla/webcam/quickstart.pyIDS
| Script | What it covers |
|---|---|
capture_image_example.py | Captures a single image from an IDS camera. |
capture_video_example.py | Captures a video stream from an IDS camera. |
camera_loop.py | Continuously reads frames from an IDS camera in a loop. |
publish_video_example.py | Publishes an IDS camera video stream over BabyROS. |
advanced_example.py | Demonstrates advanced IDS camera configuration and capture. |
python examples/medulla/ids/capture_image_example.pySynapse
Each script is run directly with:
python examples/synapse/<script_or_subdir/script>.pyQuickstart
| Script | What it covers |
|---|---|
quickstart_set_cartesian_pose_universal_robots.py | Moves a Universal Robots arm to a target Cartesian pose. |
quickstart_set_cartesian_pose_franka_robotics.py | Moves a Franka Robotics arm to a target Cartesian pose. |
quickstart_set_cartesian_pose_abb.py | Moves an ABB robot arm to a target Cartesian pose. |
quickstart_set_cartesian_pose_kuka.py | Moves a KUKA robot arm to a target Cartesian pose. |
quickstart_set_cartesian_pose_fanuc.py | Moves a Fanuc robot arm to a target Cartesian pose. |
quickstart_set_cartesian_pose_motoman.py | Moves a Yaskawa Motoman arm to a target Cartesian pose. |
quickstart_set_cartesian_pose_neura_robotics.py | Moves a Neura Robotics arm to a target Cartesian pose. |
quickstart_set_joint_positions_universal_robots.py | Commands a Universal Robots arm to specific joint positions. |
quickstart_set_joint_positions_franka_robotics.py | Commands a Franka Robotics arm to specific joint positions. |
quickstart_set_joint_positions_abb.py | Commands an ABB robot arm to specific joint positions. |
quickstart_set_joint_positions_kuka.py | Commands a KUKA robot arm to specific joint positions. |
quickstart_set_joint_positions_fanuc.py | Commands a Fanuc robot arm to specific joint positions. |
quickstart_set_joint_positions_motoman.py | Commands a Yaskawa Motoman arm to specific joint positions. |
quickstart_set_joint_positions_neura_robotics.py | Commands a Neura Robotics arm to specific joint positions. |
python examples/synapse/quickstart_set_cartesian_pose_universal_robots.pyConnection and Disconnection
| Script | What it covers |
|---|---|
connection_and_disconnection/connection_and_disconnection.py | Demonstrates connecting to and disconnecting from a robot. |
python examples/synapse/connection_and_disconnection/connection_and_disconnection.pyDiagnostics
| Script | What it covers |
|---|---|
diagnostics/get_speed_scaling_combined.py | Reads the combined speed scaling factor applied by the controller. |
diagnostics/get_target_speed_fraction.py | Reads the target speed fraction set for the robot program. |
python examples/synapse/diagnostics/get_speed_scaling_combined.pyForce Control
| Script | What it covers |
|---|---|
force_control/is_tool_in_contact.py | Detects whether the robot tool is currently in contact with an object. |
python examples/synapse/force_control/is_tool_in_contact.pyKinematics
| Script | What it covers |
|---|---|
kinematics/setup_kinematics_solver.py | Initializes and configures the kinematics solver for a robot model. |
kinematics/forward_kinematics.py | Computes end-effector pose from a given set of joint positions. |
kinematics/inverse_kinematics.py | Solves joint positions for a target Cartesian end-effector pose. |
kinematics/inverse_kinematics_with_seed.py | Solves IK starting from a user-provided seed configuration. |
kinematics/inverse_kinematics_with_profile.py | Solves IK with a motion profile for smoother trajectory planning. |
kinematics/inverse_kinematics_collision_free.py | Finds a collision-free IK solution for a target Cartesian pose. |
kinematics/get_link_transforms.py | Retrieves the transformation matrices for each robot link. |
kinematics/set_default_joint_configuration.py | Sets the default joint configuration used as IK seed. |
python examples/synapse/kinematics/setup_kinematics_solver.pyMotion
| Script | What it covers |
|---|---|
motion/set_joint_positions.py | Moves the robot to target joint positions. |
motion/set_joint_positions_advanced.py | Moves to joint positions with advanced motion parameters. |
motion/set_joint_positions_with_visualization_example.py | Moves to joint positions and visualizes the motion in Rerun. |
motion/set_joint_positions_advanced_with_visualization.py | Advanced joint motion with Rerun visualization. |
motion/set_cartesian_pose.py | Moves the end-effector to a target Cartesian pose. |
motion/set_cartesian_pose_advanced.py | Moves to a Cartesian pose with advanced motion parameters. |
motion/set_cartesian_pose_with_visualization_example.py | Moves to a Cartesian pose and visualizes in Rerun. |
motion/set_cartesian_pose_advanced_with_visualization.py | Advanced Cartesian motion with Rerun visualization. |
motion/set_cartesian_pose_in_joint_space.py | Reaches a Cartesian target via joint-space planning. |
motion/set_cartesian_pose_in_joint_space_advanced.py | Advanced joint-space planning to a Cartesian target. |
motion/set_joint_position_in_cartesian_space.py | Commands joint positions while tracking a Cartesian-space path. |
motion/contact_detection.py | Detects contact events during robot motion. |
motion/move_until_contact.py | Moves the robot along a direction until contact is detected. |
motion/start_and_stop_freedrive_mode.py | Enables and disables freedrive (gravity-compensated hand-guiding) mode. |
motion/start_and_stop_jog_mode.py | Enables and disables jog mode for incremental manual movement. |
motion/start_and_stop_teach_mode.py | Enables and disables teach mode for recording robot poses. |
motion/stop_cartesian_motion.py | Stops an ongoing Cartesian motion command. |
motion/stop_joint_motion.py | Stops an ongoing joint motion command. |
motion/trigger_protective_stop.py | Triggers a protective stop to safely halt the robot. |
python examples/synapse/motion/set_joint_positions.pyRobot Statuses
| Script | What it covers |
|---|---|
robot_statuses/get_robot_status.py | Reads the overall robot status. |
robot_statuses/get_robot_mode.py | Reads the current robot operating mode. |
robot_statuses/get_runtime_state.py | Reads the runtime execution state of the robot program. |
robot_statuses/get_safety_mode.py | Reads the current safety mode of the robot. |
robot_statuses/get_controller_frequency.py | Reads the control loop frequency of the robot controller. |
python examples/synapse/robot_statuses/get_robot_status.pyServo Control
| Script | What it covers |
|---|---|
servo_control/servo_joint.py | Streams joint position targets at high frequency for servo control. |
servo_control/servo_cartesian.py | Streams Cartesian pose targets at high frequency for servo control. |
servo_control/servo_circular.py | Executes a circular trajectory using servo control. |
servo_control/servo_stop.py | Stops an active servo motion. |
python examples/synapse/servo_control/servo_joint.pyState Reading
| Script | What it covers |
|---|---|
state_reading/is_connected.py | Checks whether the SDK is connected to the robot. |
state_reading/get_cartesian_pose.py | Reads the current end-effector Cartesian pose. |
state_reading/get_joint_positions.py | Reads the current joint positions. |
state_reading/get_joint_velocities.py | Reads the current joint velocities. |
state_reading/get_joint_torques.py | Reads the current joint torques. |
state_reading/get_actual_tcp_speed.py | Reads the actual TCP (tool center point) speed. |
state_reading/get_actual_tcp_force.py | Reads the actual force/torque at the TCP. |
state_reading/get_target_tcp_pose.py | Reads the target TCP pose commanded to the controller. |
state_reading/get_target_tcp_speed.py | Reads the target TCP speed commanded to the controller. |
state_reading/get_target_joint_positions.py | Reads the target joint positions commanded to the controller. |
state_reading/get_target_joint_velocities.py | Reads the target joint velocities commanded to the controller. |
state_reading/get_target_joint_accelerations.py | Reads the target joint accelerations commanded to the controller. |
state_reading/get_timestamp.py | Reads the robot controller timestamp. |
python examples/synapse/state_reading/is_connected.pyTools
| Script | What it covers |
|---|---|
tools/connnection_and_disconnection.py | Connects to and disconnects from a robot tool (e.g. gripper). |
tools/get_current_position.py | Reads the current position of a robot tool. |
tools/open.py | Opens a robot tool (e.g. gripper). |
tools/close.py | Closes a robot tool (e.g. gripper). |
tools/move.py | Moves a robot tool to a specified position. |
tools/set_speed.py | Sets the movement speed of a robot tool. |
tools/set_force.py | Sets the gripping force of a robot tool. |
tools/set_position_range.py | Configures the position range limits of a robot tool. |
tools/set_unit.py | Sets the measurement units used by a robot tool. |
python examples/synapse/tools/connnection_and_disconnection.pyVisualization and Model
| Script | What it covers |
|---|---|
visualization_and_model/get_model.py | Retrieves the robot's kinematic model. |
visualization_and_model/get_visual_model.py | Retrieves the robot's visual mesh model. |
visualization_and_model/get_collision_model.py | Retrieves the robot's collision geometry model. |
visualization_and_model/get_link_transforms.py | Retrieves transformation matrices for each robot link from the visual model. |
visualization_and_model/get_visual_meshes_data.py | Retrieves raw mesh data for all visual meshes. |
visualization_and_model/get_visual_mesh_transforms.py | Retrieves transformation matrices for all visual meshes. |
python examples/synapse/visualization_and_model/get_model.py
