Telekinesis Agentic Skill Library

Appearance

Medulla: Hardware Communication Skills ​

WARNING

All the cameras were tested on Windows 11 only.

SUMMARY

Medulla is a module in the Telekinesis SDK for connecting to cameras and sensors. It provides tools to interface with various 2D and 3D cameras, and integrate them into Telekinesis applications.

Install Telekinesis Skill Library
Create an API key, install telekinesis-ai and module specific installations and dependencies in a Python environment.
Open installation →
Go to the Quickstart
Pick a starter and run your first Skill end-to-end - 2D / 3D vision, webcam capture, robot motion, or a full vision-to-robot pipeline, all visualized in Rerun.
Open quickstart →

Overview of Skills ​

20 skills

Camera Skills 14

camera.connect()Medulla

Connect

Open a connection to a camera via the Medulla module of the Telekinesis SDK.

camera.disconnect()Medulla

Disconnect

Cleanly release a camera connection in the Medulla module of the Telekinesis SDK.

camera.capture_color_image()Medulla

Capture Color Image

Capture a single color frame from a camera using the Medulla module of the Telekinesis SDK.

camera.capture_depth_image()Medulla

Capture Depth Image

Capture a single depth frame from a 3D camera using the Medulla module of the Telekinesis SDK.

camera.capture_pointcloud()Medulla

Capture Point Cloud

Capture a colour-mapped point cloud from a 3D camera using the Medulla module of the Telekinesis SDK.

camera.capture_video_color_frame()Medulla

Stream Live Color Video

Stream continuous color frames from a camera using the Medulla module of the Telekinesis SDK.

camera.capture_video_depth_frame()Medulla

Stream Live Depth Video

Stream continuous depth frames from a 3D camera using the Medulla module of the Telekinesis SDK.

camera.capture_video_pointcloud()Medulla

Stream Point Cloud

Stream a continuous colour-mapped point cloud from a 3D camera using the Medulla module of the Telekinesis SDK.

camera.get_parameter()Medulla

Get Parameter

Read a camera parameter (exposure, frame rate, etc.) using the Medulla module of the Telekinesis SDK.

camera.set_parameter()Medulla

Set Parameter

Write a camera parameter (exposure, frame rate, etc.) using the Medulla module of the Telekinesis SDK.

camera.load_settings()Medulla

Load Settings

Load camera settings from a YAML file using the Medulla module of the Telekinesis SDK.

camera.save_settings()Medulla

Save Settings

Save camera settings to a YAML file using the Medulla module of the Telekinesis SDK.

Medulla

Get Intrinsics

Retrieve camera intrinsic parameters using the Medulla module of the Telekinesis SDK.

Medulla

Publish Video with BabyROS

Stream live video over BabyROS using a server/client split, and remotely control camera capture and parameters via BabyROS topics.

Sensor Skills 2

sensor.connect()Medulla

Connection and Disconnection

Connect to and disconnect from a lightbeam sensor prim in a running Isaac Sim stage via the Medulla module of the Telekinesis SDK.

sensor.is_beam_broken()Medulla

Is Beam Broken

Read whether a lightbeam sensor's beam is broken in a running Isaac Sim stage via the Medulla module of the Telekinesis SDK.

Conveyor Skills 2

belt.connect()Medulla

Connection and Disconnection

Connect to and disconnect from a conveyor prim in a running Isaac Sim stage via the Medulla module of the Telekinesis SDK.

belt.start()Medulla

Start and Stop

Run and stop a conveyor belt in a running Isaac Sim stage via the Medulla module of the Telekinesis SDK.

Machine Skills 2

machine.connect()Medulla

Connection and Disconnection

Connect to and disconnect from a CNC machine's door prim in a running Isaac Sim stage via the Medulla module of the Telekinesis SDK.

machine.open()Medulla

Open and Close Door

Slide a CNC machine's door open or closed in a running Isaac Sim stage via the Medulla module of the Telekinesis SDK.

When to Use Medulla? ​

Use Medulla for robotics applications that require camera and sensor connectivity, such as:

  • Streaming camera data for vision pipelines
  • Orchestrating multiple cameras for robot perception and control
  • Connecting cameras to Physical AI agents

What Does Medulla Provide? ​

Medulla includes a collection of modular skills for:

  • 2D and 3D camera interfacing
  • Data acquisition and preprocessing
  • Time-synchronized sensor streams
  • Integration with Telekinesis modules (Vitreous, Retina, Cornea, Pupil, Neuroplan)

How to Use Medulla? ​

To use the skills from Medulla, simply use:

python
from telekinesis.medulla.cameras import webcam

Here is a minimal example:

python
from telekinesis.medulla.cameras import webcam

camera = webcam.Webcam(name="my_webcam_camera", camera_id=0)
camera.connect()
image = camera.capture_color_image()
print(image)
camera.disconnect()

Sensors and conveyors follow the same connect / disconnect pattern, against a prim in a running Isaac Sim stage instead of physical hardware:

python
from telekinesis.medulla.sensors import isaacsim as lightbeam
from telekinesis.medulla.conveyors import isaacsim as conveyor

sensor = lightbeam.LightBeamSensor(name="cell_1_lightbeam")
sensor.connect(simulation_prim_path="/World/LightBeam_Sensor")

belt = conveyor.Conveyor(name="infeed_belt")
belt.connect(simulation_prim_path="/World/ConveyorBelt_A08")
belt.start()

if sensor.is_beam_broken():
    belt.stop()

sensor.disconnect()
belt.disconnect()