Axon: Camera Calibration Skills

SUMMARY

Axon is the camera calibration module in the Telekinesis SDK. From images of a chessboard, ChArUco, or ArUco target, it solves a single camera's intrinsics, the eye-in-hand transform between a wrist-mounted camera and the robot TCP, and the extrinsics between N synchronized cameras.

Theory

Every calibrator in Axon follows the same pattern: detect a known Target in a set of images, then solve for the unknown geometric quantity that best explains those detections.

Installation

pip install telekinesis-axon

Run the Calibration

Pick a manipulator and camera combination to see a data-collection sweep in action.

Universal Robots UR10e with a wrist-mounted Sensopart Visor camera

Universal Robots UR10e with a wrist mounted RealSense D455i camera

Epson manipulator with a wrist-mounted RealSense D405 camera

The Code

Minimal robot-driven data collection and calibration of an eye-in-hand camera. Copy paste the below code and and replace the robot and cameras to suit your hardware setup.

"""
Minimal robot-driven data collection and calibration of an eye-in-hand camera.

The robot moves the camera around a ChArUco target, saves the images and corresponding
robot poses to disk, and solves tcp_T_camera with EyeInHandCalibrator.

Output:
    data/cam_00/image_NN.png
    data/cam_00/calibration_data.npz  ← robot_T_tcp_list
"""

import argparse
from pathlib import Path

from telekinesis.medulla.cameras import realsense
from telekinesis.synapse.robots.manipulators import universal_robots

from telekinesis import axon
from telekinesis.axon import targets


def parse_args():
    """
    Parse command line arguments for the data collection script.
    """
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--robot-ip", required=True)
    parser.add_argument("--camera-serial", required=True)
    parser.add_argument("--data-dir", type=Path, required=True)
    parser.add_argument(
        "--home-pose-deg",
        type=float,
        nargs=6,
        required=True,
        metavar=("X", "Y", "Z", "RX", "RY", "RZ"),
        help="Robot home pose the sampler perturbs around",
    )
    parser.add_argument(
        "--rotation-deg", type=float, default=15.0, help="Max rotation per axis"
    )
    parser.add_argument(
        "--translation-m", type=float, default=0.05, help="Max translation per axis"
    )
    parser.add_argument("--rotation-steps", type=int, default=3)
    parser.add_argument("--translation-steps", type=int, default=2)
    parser.add_argument("--squares-x", type=int, default=6)
    parser.add_argument("--squares-y", type=int, default=9)
    parser.add_argument("--square-length", type=float, default=0.012, help="meters")
    parser.add_argument("--marker-length", type=float, default=0.009, help="meters")
    parser.add_argument("--aruco-dict-id", default="DICT_4X4_50")
    parser.add_argument(
        "--wipe", action="store_true", help="Delete existing data-dir before collecting"
    )
    return parser.parse_args()


def main(args):
    """
    Collect calibration data.

    1. Define the ChArUco target and perturbation sampler.
    2. Connect to the camera and robot.
    3. Move the robot to a series of poses and capture, and save images using DataCollector.
    4. Disconnect from the camera and robot.
    """

    # Define the ChArUco target
    target = targets.CharucoTarget(
        squares_x=args.squares_x,
        squares_y=args.squares_y,
        square_length=args.square_length,
        marker_length=args.marker_length,
        aruco_dict_id=args.aruco_dict_id,
    )

    # Define the perturbation sampler to generate robot poses around the home pose
    sampler = axon.PerturbationSampler(
        args.home_pose_deg,
        rotation_deg=args.rotation_deg,
        translation_m=args.translation_m,
        rotation_steps=args.rotation_steps,
        translation_steps=args.translation_steps,
    )

    # Define the camera and robot
    cam = realsense.RealSense(name="cam_00", serial_number=args.camera_serial)
    robot = universal_robots.UniversalRobotsUR10E(name="robot")

    try:
        # Connect to the camera and robot
        cam.connect()
        robot.connect(ip=args.robot_ip)

        # Start collection
        collector = axon.DataCollector(robot, [cam], target, sampler)
        result = collector.collect(args.data_dir, wipe=args.wipe)

        print(f"{'Done' if result['ok'] else 'Warning: <4 frames'} — data in {args.data_dir}")

        # Solve tcp_T_camera from the collected poses and images
        calibrator = axon.EyeInHandCalibrator(target)
        calib_result = calibrator.calibrate(
            robot_T_tcp_list=result["robot_T_tcp_list"],
            image_list=result["images"][0],
            method="TSAI",
        )
        print(f"Calibration {'succeeded' if calib_result.ok else 'failed'}")
        print(calib_result.tcp_T_camera)

    except KeyboardInterrupt:
        # Frames already written to data-dir are kept -- collect() writes as it goes
        print(f"\nInterrupted — stopping collection. Partial data in {args.data_dir}")

    finally:
        # Disconnect
        cam.disconnect()
        robot.disconnect()
        robot.shutdown()


if __name__ == "__main__":
    main(parse_args())