Skip to content

Telekinesis Skill Examples

Free TierEvery new account starts on a free tier with credits to call the SDK — no billing details required.Create API key →

Prerequisites

New here? Complete the Quickstart first — it sets up your API key, conda environment, and SDK in under 5 minutes.

Run an Example!

Run any example from telekinesis-examples repository

Clone the repository and the telekinesis-data submodule

bash
git clone --depth 1 --recurse-submodules --shallow-submodules \
  https://github.com/telekinesis-ai/telekinesis-examples.git

cd telekinesis-examples
What gets downloaded?

The flags above keep the clone small:

  • --depth 1 fetches only the latest commit, not full history.
  • --recurse-submodules --shallow-submodules also pulls the telekinesis-data submodule (sample images, point clouds, robot states), again at depth 1. Roughly 1.5 GB of free disk space is required for the telekinesis-data.

When you start using Telekinesis on your own data, you can ignore the telekinesis-data submodule - it exists purely to make the examples runnable out of the box.

Install dependencies for examples

bash
conda activate telekinesis
pip install rerun-sdk==0.31.2 pycocotools scipy

Run an example (see more examples below)

Runs segment_image_using_sam and visualizes masks in Rerun.

bash
python examples/cornea_examples.py --example segment_image_using_sam

SAM segmentation example: input image on the left, segmentation masks on the right

SAM masks predicted from the sample input image.

Cornea

List all available examples and run one:

bash
python examples/cornea_examples.py --list
bash
python examples/cornea_examples.py --example <name>
Example nameWhat it covers
segment_image_using_rgbSegments regions by thresholding raw RGB color channels.
segment_image_using_hsvSegments regions by thresholding hue, saturation, and value.
segment_image_using_labSegments regions using perceptually uniform LAB color space.
segment_image_using_ycrcbSegments regions by separating luminance from chroma components.
segment_image_using_focus_regionIsolates a rectangular region of interest for downstream processing.
segment_image_using_watershedSegments touching or overlapping objects using the watershed algorithm.
segment_image_using_flood_fillSegments a contiguous region by flood-filling from a seed point.
segment_image_using_foreground_birefnetExtracts foreground from background using the BiRefNet deep learning model.
segment_image_using_samSegments arbitrary objects using Meta's Segment Anything Model (SAM).
segment_image_using_grab_cutSeparates foreground from background using the GrabCut graph-cut algorithm.
segment_image_using_felzenszwalbOver-segments an image into superpixels using Felzenszwalb's method.
segment_image_using_slic_superpixelGroups pixels into compact superpixels using the SLIC algorithm.
filter_segments_by_areaRemoves segments outside a specified minimum or maximum area.
filter_segments_by_colorKeeps only segments whose average color falls within a target range.
filter_segments_by_maskFilters segments based on overlap with a provided binary mask.
segment_image_using_otsu_thresholdBinarizes an image by automatically finding the optimal Otsu threshold.
segment_image_using_local_thresholdApplies adaptive thresholding over local image neighborhoods.
segment_image_using_yen_thresholdBinarizes an image using Yen's maximum correlation thresholding method.
segment_image_using_thresholdSegments an image using a fixed manually specified threshold value.
segment_image_using_adaptive_thresholdComputes per-pixel thresholds based on local mean or Gaussian-weighted neighborhood.
segment_image_using_laplacian_thresholdDetects and segments edges by thresholding the Laplacian of the image.
bash
python examples/cornea_examples.py --example segment_image_using_rgb

Retina

List all available examples and run one:

bash
python examples/retina_examples.py --list
bash
python examples/retina_examples.py --example <name>
Example nameWhat it covers
detect_circle_using_classic_houghDetects circles in an image using the classical Hough Circle Transform.
detect_contoursFinds and draws object contours using edge-based detection.
detect_objects_using_yoloxDetects objects using the YOLOX neural network model.
detect_objects_using_rfdetrDetects objects using the RF-DETR transformer-based detection model.
detect_objects_using_qwenDetects and describes objects using the Qwen vision-language model.
detect_objects_using_grounding_dinoDetects objects from free-text prompts using Grounding DINO.
bash
python examples/retina_examples.py --example detect_contours

Pupil

List all available examples and run one:

bash
python examples/pupil_examples.py --list
bash
python examples/pupil_examples.py --example <name>
Example nameWhat it covers
filter_image_using_morphological_erodeShrinks bright regions by eroding pixel boundaries.
filter_image_using_morphological_dilateExpands bright regions by dilating pixel boundaries.
filter_image_using_morphological_closeFills small holes and gaps by closing morphological regions.
filter_image_using_morphological_openRemoves small noise while preserving larger structures.
filter_image_using_morphological_gradientHighlights edges by computing the morphological gradient.
filter_image_using_morphological_tophatExtracts small bright features against a dark background.
filter_image_using_morphological_blackhatExtracts small dark features against a bright background.
filter_image_using_morphological_hitmissDetects specific pixel patterns using hit-or-miss morphology.
filter_image_using_frangiEnhances tubular and vessel-like structures using the Frangi filter.
filter_image_using_hessianHighlights blob and ridge structures using Hessian-based filtering.
filter_image_using_satoDetects tubular structures using the Sato vesselness filter.
filter_image_using_meijeringEnhances neurite-like ridges using the Meijering filter.
filter_image_using_laplacianSharpens edges by applying the Laplacian second-derivative filter.
filter_image_using_sobelDetects edges by computing the Sobel gradient magnitude.
filter_image_using_scharrDetects edges with improved rotational accuracy using the Scharr operator.
filter_image_using_gaborExtracts texture features at specific orientations and frequencies using a Gabor filter.
filter_image_using_bilateralSmooths an image while preserving sharp edges using bilateral filtering.
filter_image_using_boxApplies a uniform box blur to smooth the image.
filter_image_using_gaussian_blurSmooths an image using a Gaussian kernel to reduce noise.
filter_image_using_median_blurRemoves salt-and-pepper noise using median filtering.
filter_image_using_blurApplies a simple average blur to the image.
enhance_image_using_auto_gamma_correctionAutomatically adjusts image brightness using gamma correction.
enhance_image_using_white_balanceCorrects color casts by adjusting white balance.
enhance_image_using_claheImproves local contrast using Contrast Limited Adaptive Histogram Equalization.
transform_image_using_pyramid_downsamplingReduces image resolution using Gaussian pyramid downsampling.
transform_image_using_pyramid_upsamplingIncreases image resolution using Laplacian pyramid upsampling.
transform_mask_using_blob_thinningSkeletonizes binary mask blobs to single-pixel-wide lines.
calculate_image_pcaComputes principal components of pixel data for dimensionality analysis.
calculate_image_centroidCalculates the centroid of intensity or binary regions in an image.
convert_image_color_spaceConverts an image between color spaces (e.g. BGR to HSV).
normalize_image_intensityRescales pixel intensities to a standard range.
split_image_into_channelsSplits a multi-channel image into individual channel images.
merge_image_from_channelsMerges separate channel images into a single multi-channel image.
resize_imageResizes an image to a specified width and height.
resize_image_with_aspect_fitResizes an image while preserving its aspect ratio.
rotate_imageRotates an image by a given angle around its center.
translate_imageShifts an image by a specified pixel offset.
pad_imageAdds padding around the image borders.
crop_image_centerCrops a region from the center of an image.
crop_image_using_bounding_boxesCrops regions from an image using bounding box coordinates.
crop_image_using_polygonCrops and masks a region defined by a polygon.
bitwise_and_imagesCombines two images using a bitwise AND operation.
bitwise_or_imagesCombines two images using a bitwise OR operation.
bitwise_xor_imagesHighlights differences between two images using bitwise XOR.
bitwise_difference_imagesComputes the absolute pixel difference between two images.
bitwise_not_imageInverts pixel values using a bitwise NOT operation.
overlay_images_using_weighted_overlayBlends two images together using configurable weights.
project_pixel_to_camera_pointBack-projects a 2D pixel to a 3D camera-frame point using depth.
project_camera_point_to_pixelProjects a 3D camera-frame point onto the 2D image plane.
project_pixel_to_world_pointBack-projects a 2D pixel to a 3D world-frame point using depth and pose.
project_world_point_to_pixelProjects a 3D world-frame point onto the 2D image plane.
bash
python examples/pupil_examples.py --example filter_image_using_sobel

Vitreous

List all available examples and run one:

bash
python examples/vitreous_examples.py --list
bash
python examples/vitreous_examples.py --example <name>
Example nameWhat it covers
calculate_axis_aligned_bounding_boxComputes the axis-aligned bounding box of a point cloud.
calculate_oriented_bounding_boxComputes the tightest oriented bounding box around a point cloud.
calculate_point_cloud_centroidCalculates the geometric centroid of a point cloud.
calculate_points_in_point_cloudReturns the total number of points in a point cloud.
calculate_plane_normalEstimates the normal vector of a planar surface in a point cloud.
cluster_point_cloud_using_dbscanGroups points into clusters using the DBSCAN density-based algorithm.
cluster_point_cloud_based_on_density_jumpSegments clusters by detecting jumps in local point density.
convert_mesh_to_point_cloudSamples a point cloud from the surface of a 3D mesh.
create_cylinder_meshCreates a parametric cylinder mesh.
create_plane_meshCreates a parametric plane mesh.
create_sphere_meshCreates a parametric sphere mesh.
create_torus_meshCreates a parametric torus mesh.
estimate_principal_axis_within_radiusEstimates the principal axis of points within a specified radius.
estimate_principal_axesComputes the principal axes of an entire point cloud using PCA.
filter_point_cloud_using_pass_through_filterRemoves points outside a specified range along one axis.
filter_point_cloud_using_bounding_boxKeeps only points within an axis-aligned bounding box.
filter_point_cloud_using_cylinder_base_removalRemoves points belonging to a detected cylinder base.
filter_point_cloud_using_maskFilters points using a boolean mask array.
filter_point_cloud_using_oriented_bounding_boxKeeps only points within an oriented bounding box.
filter_point_cloud_using_plane_proximityRetains points within a distance threshold from a fitted plane.
filter_point_cloud_using_plane_defined_by_point_normal_proximityRetains points near a plane defined by a point and normal vector.
filter_point_cloud_using_plane_splittingSplits a point cloud into two halves along a plane.
filter_point_cloud_using_radius_outlier_removalRemoves points with too few neighbors within a given radius.
filter_point_cloud_using_statistical_outlier_removalRemoves statistical outliers based on mean distance to neighbors.
filter_point_cloud_using_uniform_downsamplingReduces point count by keeping every nth point uniformly.
filter_point_cloud_using_viewpoint_visibilityRemoves points not visible from a specified camera viewpoint.
filter_point_cloud_using_voxel_downsamplingDownsamples a point cloud by averaging points within voxel cells.
add_point_cloudsMerges two point clouds into one.
subtract_point_cloudsRemoves points from one cloud that overlap with another.
scale_point_cloudScales a point cloud by a given factor.
apply_transform_to_point_cloudApplies a rigid 4×4 transformation matrix to a point cloud.
project_point_cloud_to_planeProjects all points onto a fitted plane.
project_point_cloud_to_plane_defined_by_point_normalProjects all points onto a plane defined by a point and normal.
reconstruct_mesh_using_convex_hullReconstructs a surface mesh as the convex hull of a point cloud.
reconstruct_mesh_using_poissonReconstructs a watertight surface mesh using Poisson reconstruction.
register_point_clouds_using_cuboid_translation_sampler_icpAligns point clouds using ICP with a cuboid translation sampler initialization.
register_point_clouds_using_fast_global_registrationAligns point clouds using Fast Global Registration for coarse alignment.
register_point_clouds_using_point_to_plane_icpRefines point cloud alignment using point-to-plane ICP.
register_point_clouds_using_point_to_point_icpRefines point cloud alignment using point-to-point ICP.
register_point_clouds_using_rotation_sampler_icpAligns point clouds using ICP with a rotation sampler initialization.
segment_point_cloud_using_colorSegments a point cloud into regions by color similarity.
segment_point_cloud_using_planeSegments the dominant plane from a point cloud using RANSAC.
bash
python examples/vitreous_examples.py --example scale_point_cloud

Datatypes

List all available examples and run one:

bash
python examples/datatypes_examples.py --list
bash
python examples/datatypes_examples.py --example <name>
Example nameWhat it covers
boolDemonstrates the Telekinesis boolean datatype.
floatDemonstrates the Telekinesis float datatype.
intDemonstrates the Telekinesis integer datatype.
float32Demonstrates the 32-bit float datatype.
stringDemonstrates the Telekinesis string datatype.
rgba32Demonstrates the RGBA32 color datatype.
imageDemonstrates the Telekinesis image datatype.
imageformatDemonstrates supported image format encodings.
mat3x3Demonstrates the 3×3 matrix datatype.
mat4x4Demonstrates the 4×4 transformation matrix datatype.
vector3dDemonstrates the 3D vector datatype.
vector4dDemonstrates the 4D vector datatype.
points2dDemonstrates the 2D point array datatype.
points3dDemonstrates the 3D point array datatype.
points3d_listDemonstrates a list of 3D point arrays.
boxes2dDemonstrates the 2D bounding box datatype.
boxes3dDemonstrates the 3D bounding box datatype.
mesh3dDemonstrates the 3D mesh datatype.
linestrips2dDemonstrates the 2D line strip datatype.
circlesDemonstrates the circle geometry datatype.
ellipsesDemonstrates the ellipse geometry datatype.
linesDemonstrates the line geometry datatype.
channeldatatypeDemonstrates the channel data type descriptor.
colormodelDemonstrates the color model datatype.
categoriesDemonstrates the category label datatype.
object_detection_annotationsDemonstrates bounding-box object detection annotation format.
panoptic_segmentation_annotationDemonstrates panoptic segmentation annotation format.
geometry_detection_annotationsDemonstrates geometric shape detection annotation format.
partDemonstrates the part datatype for object decomposition.
trayDemonstrates the tray datatype for bin-picking scenes.
binDemonstrates the bin datatype for bin-picking scenes.
bash
python examples/datatypes_examples.py --example bool

Medulla

Each script is run directly with:

bash
python examples/medulla/<vendor>/<script>.py

Webcam

ScriptWhat it covers
quickstart.pyMinimal example to capture a frame from a webcam.
capture_image_example.pyCaptures a single image from a webcam.
capture_video_example.pyCaptures a video stream from a webcam.
camera_loop.pyContinuously reads frames from a webcam in a loop.
publish_video_example.pyPublishes a webcam video stream over BabyROS.
bash
python examples/medulla/webcam/quickstart.py

IDS

ScriptWhat it covers
capture_image_example.pyCaptures a single image from an IDS camera.
capture_video_example.pyCaptures a video stream from an IDS camera.
camera_loop.pyContinuously reads frames from an IDS camera in a loop.
publish_video_example.pyPublishes an IDS camera video stream over BabyROS.
advanced_example.pyDemonstrates advanced IDS camera configuration and capture.
bash
python examples/medulla/ids/capture_image_example.py

Synapse

Each script is run directly with:

bash
python examples/synapse/<script_or_subdir/script>.py

Quickstart

ScriptWhat it covers
quickstart_set_cartesian_pose_universal_robots.pyMoves a Universal Robots arm to a target Cartesian pose.
quickstart_set_cartesian_pose_franka_robotics.pyMoves a Franka Robotics arm to a target Cartesian pose.
quickstart_set_cartesian_pose_abb.pyMoves an ABB robot arm to a target Cartesian pose.
quickstart_set_cartesian_pose_kuka.pyMoves a KUKA robot arm to a target Cartesian pose.
quickstart_set_cartesian_pose_fanuc.pyMoves a Fanuc robot arm to a target Cartesian pose.
quickstart_set_cartesian_pose_motoman.pyMoves a Yaskawa Motoman arm to a target Cartesian pose.
quickstart_set_cartesian_pose_neura_robotics.pyMoves a Neura Robotics arm to a target Cartesian pose.
quickstart_set_joint_positions_universal_robots.pyCommands a Universal Robots arm to specific joint positions.
quickstart_set_joint_positions_franka_robotics.pyCommands a Franka Robotics arm to specific joint positions.
quickstart_set_joint_positions_abb.pyCommands an ABB robot arm to specific joint positions.
quickstart_set_joint_positions_kuka.pyCommands a KUKA robot arm to specific joint positions.
quickstart_set_joint_positions_fanuc.pyCommands a Fanuc robot arm to specific joint positions.
quickstart_set_joint_positions_motoman.pyCommands a Yaskawa Motoman arm to specific joint positions.
quickstart_set_joint_positions_neura_robotics.pyCommands a Neura Robotics arm to specific joint positions.
bash
python examples/synapse/quickstart_set_cartesian_pose_universal_robots.py

Connection and Disconnection

ScriptWhat it covers
connection_and_disconnection/connection_and_disconnection.pyDemonstrates connecting to and disconnecting from a robot.
bash
python examples/synapse/connection_and_disconnection/connection_and_disconnection.py

Diagnostics

ScriptWhat it covers
diagnostics/get_speed_scaling_combined.pyReads the combined speed scaling factor applied by the controller.
diagnostics/get_target_speed_fraction.pyReads the target speed fraction set for the robot program.
bash
python examples/synapse/diagnostics/get_speed_scaling_combined.py

Force Control

ScriptWhat it covers
force_control/is_tool_in_contact.pyDetects whether the robot tool is currently in contact with an object.
bash
python examples/synapse/force_control/is_tool_in_contact.py

Kinematics

ScriptWhat it covers
kinematics/setup_kinematics_solver.pyInitializes and configures the kinematics solver for a robot model.
kinematics/forward_kinematics.pyComputes end-effector pose from a given set of joint positions.
kinematics/inverse_kinematics.pySolves joint positions for a target Cartesian end-effector pose.
kinematics/inverse_kinematics_with_seed.pySolves IK starting from a user-provided seed configuration.
kinematics/inverse_kinematics_with_profile.pySolves IK with a motion profile for smoother trajectory planning.
kinematics/inverse_kinematics_collision_free.pyFinds a collision-free IK solution for a target Cartesian pose.
kinematics/get_link_transforms.pyRetrieves the transformation matrices for each robot link.
kinematics/set_default_joint_configuration.pySets the default joint configuration used as IK seed.
bash
python examples/synapse/kinematics/setup_kinematics_solver.py

Motion

ScriptWhat it covers
motion/set_joint_positions.pyMoves the robot to target joint positions.
motion/set_joint_positions_advanced.pyMoves to joint positions with advanced motion parameters.
motion/set_joint_positions_with_visualization_example.pyMoves to joint positions and visualizes the motion in Rerun.
motion/set_joint_positions_advanced_with_visualization.pyAdvanced joint motion with Rerun visualization.
motion/set_cartesian_pose.pyMoves the end-effector to a target Cartesian pose.
motion/set_cartesian_pose_advanced.pyMoves to a Cartesian pose with advanced motion parameters.
motion/set_cartesian_pose_with_visualization_example.pyMoves to a Cartesian pose and visualizes in Rerun.
motion/set_cartesian_pose_advanced_with_visualization.pyAdvanced Cartesian motion with Rerun visualization.
motion/set_cartesian_pose_in_joint_space.pyReaches a Cartesian target via joint-space planning.
motion/set_cartesian_pose_in_joint_space_advanced.pyAdvanced joint-space planning to a Cartesian target.
motion/set_joint_position_in_cartesian_space.pyCommands joint positions while tracking a Cartesian-space path.
motion/contact_detection.pyDetects contact events during robot motion.
motion/move_until_contact.pyMoves the robot along a direction until contact is detected.
motion/start_and_stop_freedrive_mode.pyEnables and disables freedrive (gravity-compensated hand-guiding) mode.
motion/start_and_stop_jog_mode.pyEnables and disables jog mode for incremental manual movement.
motion/start_and_stop_teach_mode.pyEnables and disables teach mode for recording robot poses.
motion/stop_cartesian_motion.pyStops an ongoing Cartesian motion command.
motion/stop_joint_motion.pyStops an ongoing joint motion command.
motion/trigger_protective_stop.pyTriggers a protective stop to safely halt the robot.
bash
python examples/synapse/motion/set_joint_positions.py

Robot Statuses

ScriptWhat it covers
robot_statuses/get_robot_status.pyReads the overall robot status.
robot_statuses/get_robot_mode.pyReads the current robot operating mode.
robot_statuses/get_runtime_state.pyReads the runtime execution state of the robot program.
robot_statuses/get_safety_mode.pyReads the current safety mode of the robot.
robot_statuses/get_controller_frequency.pyReads the control loop frequency of the robot controller.
bash
python examples/synapse/robot_statuses/get_robot_status.py

Servo Control

ScriptWhat it covers
servo_control/servo_joint.pyStreams joint position targets at high frequency for servo control.
servo_control/servo_cartesian.pyStreams Cartesian pose targets at high frequency for servo control.
servo_control/servo_circular.pyExecutes a circular trajectory using servo control.
servo_control/servo_stop.pyStops an active servo motion.
bash
python examples/synapse/servo_control/servo_joint.py

State Reading

ScriptWhat it covers
state_reading/is_connected.pyChecks whether the SDK is connected to the robot.
state_reading/get_cartesian_pose.pyReads the current end-effector Cartesian pose.
state_reading/get_joint_positions.pyReads the current joint positions.
state_reading/get_joint_velocities.pyReads the current joint velocities.
state_reading/get_joint_torques.pyReads the current joint torques.
state_reading/get_actual_tcp_speed.pyReads the actual TCP (tool center point) speed.
state_reading/get_actual_tcp_force.pyReads the actual force/torque at the TCP.
state_reading/get_target_tcp_pose.pyReads the target TCP pose commanded to the controller.
state_reading/get_target_tcp_speed.pyReads the target TCP speed commanded to the controller.
state_reading/get_target_joint_positions.pyReads the target joint positions commanded to the controller.
state_reading/get_target_joint_velocities.pyReads the target joint velocities commanded to the controller.
state_reading/get_target_joint_accelerations.pyReads the target joint accelerations commanded to the controller.
state_reading/get_timestamp.pyReads the robot controller timestamp.
bash
python examples/synapse/state_reading/is_connected.py

Tools

ScriptWhat it covers
tools/connnection_and_disconnection.pyConnects to and disconnects from a robot tool (e.g. gripper).
tools/get_current_position.pyReads the current position of a robot tool.
tools/open.pyOpens a robot tool (e.g. gripper).
tools/close.pyCloses a robot tool (e.g. gripper).
tools/move.pyMoves a robot tool to a specified position.
tools/set_speed.pySets the movement speed of a robot tool.
tools/set_force.pySets the gripping force of a robot tool.
tools/set_position_range.pyConfigures the position range limits of a robot tool.
tools/set_unit.pySets the measurement units used by a robot tool.
bash
python examples/synapse/tools/connnection_and_disconnection.py

Visualization and Model

ScriptWhat it covers
visualization_and_model/get_model.pyRetrieves the robot's kinematic model.
visualization_and_model/get_visual_model.pyRetrieves the robot's visual mesh model.
visualization_and_model/get_collision_model.pyRetrieves the robot's collision geometry model.
visualization_and_model/get_link_transforms.pyRetrieves transformation matrices for each robot link from the visual model.
visualization_and_model/get_visual_meshes_data.pyRetrieves raw mesh data for all visual meshes.
visualization_and_model/get_visual_mesh_transforms.pyRetrieves transformation matrices for all visual meshes.
bash
python examples/synapse/visualization_and_model/get_model.py

Where to Go Next

Explore the Docs

Support