Stream Point Cloud

SUMMARY

Stream Point Cloud continuously fuses depth and colour frames into colour-mapped 3D point clouds, returning each one as an object with .positions (Nx3 float XYZ) and .colors (Nx3 uint8 RGB) arrays. Configure resolution and frame rate at connect time, then log each cloud directly with rr.Points3D.

Available on: Intel RealSense.

The Skill

point_cloud = camera.capture_video_pointcloud()

Call repeatedly inside a loop. Stream parameters are configured via connect:

camera.connect(width=640, height=480, fps=30)

Example

The Code

"""
Stream a continuous colour-mapped point cloud from a RealSense camera.

Run from a terminal to avoid issues with Rerun's spawn mode.
"""
import time
from loguru import logger

import rerun as rr

from telekinesis.medulla.cameras import realsense


def main():
    camera = realsense.RealSense(name="my_realsense_camera")
    try:
        rr.init("RealSense_Pointcloud_Video_Example", spawn=True)
        rr.log("/", rr.ViewCoordinates.RDF, static=True)
        camera.connect(width=640, height=480, fps=30)

        end_time = time.time() + 10
        while time.time() < end_time:
            point_cloud = camera.capture_video_pointcloud()
            if point_cloud is None:
                continue

            rr.log(
                "Continuous_Pointcloud_Capture",
                rr.Points3D(positions=point_cloud.positions, colors=point_cloud.colors),
            )
    except Exception as e:
        logger.error(f"Unable to stream point cloud. {type(e).__name__}: {e}")
    finally:
        camera.disconnect()


if __name__ == "__main__":
    main()

Running the Example

python realsense_pointcloud_example.py

A Rerun viewer opens automatically. The 3D point cloud streams under Continuous_Pointcloud_Capture for 10 seconds, then the camera disconnects cleanly.

Parameter Configuration

capture_video_pointcloud() itself takes no parameters. Stream resolution and frame rate are configured up front via connect:

Parameter	Type	Default	Description
width	int | None	None	Stream width in pixels. Must be provided together with height and fps, or all three omitted for camera defaults.
height	int | None	None	Stream height in pixels. Must be provided together with width and fps, or all three omitted for camera defaults.
fps	int | None	None	Target frame rate. Must be provided together with width and height, or all three omitted for camera defaults.

Returns

Type	Description
datatypes.Points3D	Colour-mapped point cloud with .positions (float32, shape (N, 3), XYZ in metres) and .colors (uint8, shape (N, 4), RGBA).
None	Returned if a frame could not be retrieved for that call.

Raises

Exception	Condition
RuntimeError	The camera is not connected (the RealSense pipeline has not been started).
ArgumentError	Raised by connect(), not capture_video_pointcloud() — only some of width, height, fps were given, or the requested resolution/frame-rate combination is not supported by the depth or colour stream.

Where to Use the Skill

• Live 3D monitoring — observe how the scene's 3D structure changes during robot motion.
• Continuous grasp planning — feed a steady stream of point clouds into pose-estimation pipelines.
• Calibration sanity checks — verify depth-colour alignment in real time.