What is Telekinesis? ​

Robotics development is fragmented across incompatible tools, APIs, and frameworks. Teams must constantly integrate perception stacks, motion libraries, hardware interfaces, and AI models—resulting in brittle systems and slow iteration.

Telekinesis replaces this fragmentation with a unified library where Physical AI Agents, perception, planning, control, and robot learning work seamlessly together.

Physical AI Agent: Prompt To Robot Code ​

Prompt:

I have a UR10e and an RG6 gripper, I want to do a repackaging task where the parts are placed in a rectangular grid and need to be placed into
another grid where there is a fixed offset on the x axis and the y axis and every other row is offset from the previous row. The first row has
n slots, second m third n etc. Every other row is identical. When picking up the parts do not open the gripper all the way as the parts are close.
Start and end the program at a home position.

Getting Started ​

New to Telekinesis? This is the recommended setup path. Install the core Telekinesis Skill Library, run the Quickstart to verify your setup, then add Physical AI Agents or BabyROS when you are ready to build larger workflows.

1. Install Telekinesis Skill Library
   Create your API key, set up Python environment, and install the telekinesis-ai SDK.

2. Run the Quickstart
   Run your first vision, hardware, or robotics Skill in minutes — no clone required.

3. Install Physical AI Agents
   Once telekinesis-ai is set up, install the agent tooling to generate executable robot code from natural-language prompts.

4. Install BabyROS
   Additionally, install BabyROS to connect robots, cameras, sensors, and distributed services.

Understanding the Telekinesis Ecosystem ​

Telekinesis is a unified robotics intelligence stack designed around 5 tightly integrated layers: Skills, Agents, Data Engine, BabyROS Middleware, and Industrial Applications.

Together, they form a closed-loop system for building, executing, and improving Physical AI:

1. Skills — production-grade atomic actions for perception, planning, control, and hardware. The executable building blocks of every robot behavior.
2. Physical AI Agents — VLM/LLM-powered systems that decompose high-level goals into structured sequences of Skills.
3. Data Engine — the memory layer: captures every Skill execution (inputs, outputs, trajectories, sensor data) and turns it into usable datasets.
4. BabyROS Middleware — ultra-low-latency pub/sub and client/server messaging that runs Skills as distributed nodes across sensors, compute, and robots.
5. Industrial Applications — real-world deployments across manufacturing, logistics, and agriculture, with the same Skills and Agents validated in simulation first.

Skills ​

Skills are reusable modular operations for perception, robotics, and decision-making that can be chained into workflows for Physical AI applications in manufacturing, logistics and more.

Skill Example 1 - Vision - segment_image_using_sam: Segmentation on an image using SAM model.

from telekinesis import cornea                                # Import Cornea - Image segmentation module

# Executing a 2D image segmentation Skill
result = cornea.segment_image_using_sam(                      # Executing Skill - `segment_image_using_sam`
    image=image,
    bboxes=[[400, 150, 1200, 450]]
)
# Access results
annotations = result.to_list()

Skill Example 2 - Robot - set_joint_positions: Set robot joint positions.

# Example 2
from telekinesis.synapse.robots.manipulators.universal_robots import UniversalRobotsUR10E # Importing the UR10E robot interface from Synapse - Robotics module

# Create and connect the robot
robot = UniversalRobotsUR10E()
robot.connect(ip="192.168.1.2")
# Execute a motion control Skill to set joint positions
robot.set_joint_positions(
    joint_positions=[0, 90, 0, -90, 0, 90],
    speed=60,
    acceleration=80,
    asynchronous=False
)
# Disconnect the robot after execution
robot.disconnect()

Robotics Skills ​

Control a wide set of different industrial and mobile robots such as Universal Robots, Anybotics and others through one consistent Python interface.

from telekinesis import synapse # robotics skills

Computer Vision Skills ​

Use production-grade computer vision Skill Groups for obstacle detection, pose estimation, point-cloud processing, and AI model training and much more.

from telekinesis import cornea            # image segmentation skills
from telekinesis import retina            # object detection skills
from telekinesis import pupil             # image processing skills
from telekinesis import vitreous          # point cloud processing skills
from telekinesis import iris              # AI model training skills
from telekinesis.medulla import cameras   # Medulla hardware communication skills

6D Pose Estimation from Point Clouds

Object Detection

Object Segmentation

Reinforcement Learning, Imitation Learning and Vision-Language-Action Model Skills ​

Train and deploy learned robot policies and behaviors for locomotion, manipulation, and control.

from telekinesis import rlbotics   # reinforcement learning skills

Synthetic Dataset Generation Skills ​

Generate photo-realistic synthetic datasets to train and validate computer vision models.

from telekinesis import illusion   # synthetic data generation skills

Synthetic Image 1

Synthetic Image 2

Synthetic Image 3

Physical AI Agents ​

Physical AI Agents turn natural-language prompts into executable robot code by orchestrating the Telekinesis Skill Library. Use Tzara — our VS Code extension — to write a prompt and get a runnable Python program back.

Prompt:

I have a UR10e and an RG6 gripper. I have parts vertically placed in a grid that need to be picked and placed horizontally
in a new grid (requires a -90-degree flip between pick and place around the y axis). Add an optional intermediate joint
pose before the flip. Add logging at each step.

Data Engine ​

The memory layer of Telekinesis — automatically captures sensor streams, robot states, transforms, and full Skill I/O from every run. Replay failures against successful runs to debug, then turn it all into production-grade datasets for training, evaluation, and continual learning.

BabyROS Middleware ​

Ultra-low-latency pub/sub and client/server messaging built on Zenoh that connects sensors, actuators, AI modules, and control loops across microcontrollers, edge devices, and the cloud. ROS-style ergonomics with a single pip install babyros — no system-wide dependencies, workspace overlays, or custom .msg files required.

pip install babyros

Example Publisher:

import babyros

publisher = babyros.node.Publisher(topic="data/topic")
publisher.publish({"data": 123})

Example Subscriber:

import babyros

def callback(message):
    print("Received:", message)

subscriber = babyros.node.Subscriber(topic="data/topic", callback=callback)

5. Industrial Applications ​

Build real-world robotics and Physical AI applications for industries such as manufacturing, automotive, aerospace and others.

Join our Discord Community to Add your Own Skills ​

The Telekinesis Agentic Skill Library is the beginning of a vibrant ecosystem. Whether you are a researcher, a hobbyist, or an industrial engineer, your work belongs here. Release your Skill, let others improve it, and see it deployed in real-world systems.