EyeInHandCalibrator

SUMMARY

EyeInHandCalibrator solves tcp_T_camera, the fixed transform between a robot's TCP (flange) and a wrist-mounted camera, from images of a static calibration board taken at a series of robot poses.

The Skill

from telekinesis.axon import EyeInHandCalibrator

The Code

Requires images of a calibration board alongside the robot's TCP pose (robot_T_tcp, as 4x4 homogeneous matrices) for each frame.

from telekinesis.axon import EyeInHandCalibrator
from telekinesis.axon.targets import CharucoTarget

target = CharucoTarget(squares_x=6, squares_y=9, square_length=0.012, marker_length=0.009)

calibrator = EyeInHandCalibrator(target)

result = calibrator.calibrate(
    robot_T_tcp_list=robot_poses,  # list of (4, 4) np.ndarray
    image_list=images,
    method="TSAI",
)

result.ok               # bool
result.tcp_T_camera     # (4, 4) np.ndarray
result.intrinsic_matrix
result.distortion_coefficients
result.reprojection_error
result.consistency       # TargetConsistencyStats, when result.has_consistency

Theory

The pose-chain problem

EyeInHandCalibrator solves an eye-in-hand configuration: the camera is mounted rigidly on the robot's wrist and moves with it, as opposed to an eye-to-hand setup where the camera is stationary and observes the robot and target from a fixed external position. Axon currently implements eye-in-hand only.

At every captured pose, three transforms are available:

• robot_T_tcp, robot base to TCP (flange), read directly from the robot controller.
• camera_T_target, camera to calibration target, estimated from the image (see IntrinsicCalibrator).
• tcp_T_camera, TCP to camera, fixed across every pose because the camera is rigidly mounted to the flange. This is what we're solving for.

A fourth transform, robot_T_target (base to target), is never measured directly, but as long as the target itself doesn't move during the capture session, it's the same at every pose:

robot_T_target = robot_T_tcp · tcp_T_camera · camera_T_target

robot_T_tcp and camera_T_target both change from pose to pose; tcp_T_camera and robot_T_target don't. Capturing at N >= 3 sufficiently distinct poses turns this into an overdetermined system with one unknown, constant transform, the classic AX = XB hand-eye formulation. calibrate() hands that system to one of five optimization methods (see Hand-Eye Methods) via cv::calibrateHandEye.

TargetConsistencyStats (the result's .consistency field) directly measures how well the robot_T_target invariant held across your captured frames, see Best Practices below.

Initialization

EyeInHandCalibrator(target, options=IntrinsicOptions())

Parameter	Type	Description
target	ChessboardTarget | CharucoTarget | ArucoTarget	Calibration target, see All Supported Targets.
options	IntrinsicOptions	Tunables for the internal intrinsic-calibration stage, see IntrinsicOptions.

Skills

Skill	Description
Calibrate	Solve tcp_T_camera from robot poses and calibration-board images.
Compute Target Poses	Precompute per-frame target poses via ChArUco detection and solvePnP.

See also EyeInHandCalibrator State to read back the last result, detection images, and target consistency stats.

Best Practices

Collecting a good dataset

• Capture 10-20+ poses. Fewer than ~10 makes the solve sensitive to noise in any single detection; more helps average it out.
• Spread rotation across all three axes, not just one. A sweep that only pans or only tilts under-constrains the rotational part of the solve, diagnostics.analyze_rotation_variety checks this directly on a captured dataset. PerturbationSampler generates this kind of varied sweep automatically.
• Match the workspace and configuration you'll actually use. Capture poses in the same region of the robot's workspace, and near the same joint configuration, as the task the calibration will feed into, hand-eye accuracy doesn't extrapolate well outside the calibrated volume.
• Keep the target fully exposed and in focus at every pose, with consistent lighting and exposure, detection quality that varies pose-to-pose adds noise the solver can't distinguish from real motion.
• With a CharucoTarget, not every marker needs to be visible in every frame, partial views are fine as long as enough corners clear min_corners (see Data Collector: collect()), but more visible markers per frame means a more accurate per-frame camera_T_target.

Common pitfalls

• The target moves during capture. The entire solve assumes robot_T_target is constant, a loose mount, a bumped board, or a target that isn't rigidly fixed will silently degrade the result rather than fail loudly. Check TargetConsistencyStats after the fact; large max_translation_m / max_rotation_deg deviations are a symptom.
• Mixed units. Axon's targets (square_length, marker_length) and poses are in meters throughout. Mixing units with a robot controller that reports translation in millimeters is a common source of a tcp_T_camera that looks structurally right but is off by 1000x in translation.
• Planar or low-diversity sweeps. A dataset where the target stays roughly parallel to the camera across all frames, pure translation, or rotation about a single axis, is numerically close to singular for the hand-eye solve. Check axis_span_ratio from analyze_rotation_variety before trusting the result; values near 0 mean the motions were close to coplanar.
• Trusting a single method. If a solve looks off, run diagnostics.compare_hand_eye_methods across all five methods. Agreement across methods is a stronger signal than a low reprojection_error from any one method alone, DANIILIDIS in particular occasionally converges to an outlier.
• Forgetting to recalibrate after remounting. tcp_T_camera is only valid as long as the physical camera-to-flange mount is undisturbed, remount the camera, and the transform is invalid until recalibrated.