Calibrate
SUMMARY
calibrate() solves tcp_T_camera, the fixed transform between a robot's TCP (flange) and a wrist-mounted camera, from images of a static calibration board taken at a series of robot poses. A closed-form hand-eye solve is followed by a joint refinement over every detected corner.
SUPPORTED TARGETS
Works with any target, ChessboardTarget, CharucoTarget, ArucoTarget, or ZividTarget, see All Supported Targets. CharucoTarget is recommended, since it gives a pose from partial views.
UNITS
Input images can be BGR, BGRA or grayscale ndarrays. robot_T_tcp_list and tcp_T_camera are (4, 4) SE(3) matrices with translation in meters. reprojection_error and the refinement RMS values are in pixels.
The Skill
result = calibrator.calibrate(
robot_T_tcp_list,
image_list,
method="TSAI",
camera_T_target_list=[],
intrinsic_matrix=None,
distortion_coefficients=None,
output_path=None,
refine=True,
)| Parameter | Type | Description |
|---|---|---|
robot_T_tcp_list | list[(4, 4) ndarray] | Per-frame robot base to TCP (or flange) transforms, aligned one-to-one with image_list. |
image_list | list[ndarray] | Per-frame images of the static board. |
method | str | Closed-form hand-eye method, see Hand-Eye Methods. Default "TSAI". |
camera_T_target_list | list[(4, 4) ndarray] | Optional pre-computed board poses, see the input modes below. |
intrinsic_matrix | (3, 3) ndarray | Optional known camera matrix. |
distortion_coefficients | ndarray | Optional known distortion coefficients. |
output_path | str | Path | None | When set, writes a hand_eye.json result file (directory or file path), see Hand-Eye Result. Default None writes nothing. |
refine | bool | Run joint reprojection refinement after the closed-form solve, see Joint Refinement. Default True. |
There are three input modes, selected by which optional arguments are supplied:
| Mode | How to trigger | Behavior |
|---|---|---|
| Full pipeline (default) | Leave intrinsic_matrix and camera_T_target_list unset | Runs intrinsic calibration on image_list, estimates a board pose on each view kept by the intrinsic fit, solves cv::calibrateHandEye, then refines. |
| Known intrinsics | Pass intrinsic_matrix (and optionally distortion_coefficients) | Skips the intrinsic stage; per-frame board poses come from target detection + planar PnP against the given K, d. Refinement runs with K fixed. |
| Strict pass-through | Pass camera_T_target_list | Skips detection entirely. robot_T_tcp_list must already be aligned to the supplied poses. No refinement, since there are no corner detections. |
calibrate() raises ValueError for an unknown method, when image_list and robot_T_tcp_list (or camera_T_target_list) have different lengths, and when an input pose is not a valid rigid 4x4 transform (orthonormal rotation, det = +1, bottom row [0, 0, 0, 1]).
The Code
Requires images of a calibration board alongside the robot's TCP pose (robot_T_tcp, as 4x4 homogeneous matrices) for each frame.
from telekinesis.axon import EyeInHandCalibrator
from telekinesis.axon.targets import CharucoTarget
target = CharucoTarget(squares_x=6, squares_y=9, square_length=0.012, marker_length=0.009)
calibrator = EyeInHandCalibrator(target)
result = calibrator.calibrate(
robot_T_tcp_list=robot_poses, # list of (4, 4) np.ndarray
image_list=images,
method="TSAI",
)
result.ok # bool
result.tcp_T_camera # (4, 4) np.ndarray, refined when refinement was accepted
result.closed_form_tcp_T_camera # (4, 4) np.ndarray, before refinement
result.intrinsic_matrix
result.distortion_coefficients
result.reprojection_error # intrinsic RMS (px)
result.refinement.rms_after_px # corner RMS through the whole robot-camera chain, when result.has_refinement
result.consistency # TargetConsistencyStats, when result.has_consistencyUse known intrinsics
If the camera is already calibrated, pass its intrinsics. Board poses are then computed per frame with that K, for any target type:
result = calibrator.calibrate(
robot_T_tcp_list=robot_poses,
image_list=images,
intrinsic_matrix=K,
distortion_coefficients=d,
)Pass pre-computed per-frame poses
If you have camera_T_target transforms from an external source, pass them via camera_T_target_list to skip detection and pose estimation (see compute_camera_T_target_list() for how to produce them). robot_T_tcp_list must be filtered to the same frames:
poses = calibrator.compute_camera_T_target_list(images, robot_poses, K, d)
result = calibrator.calibrate(
robot_T_tcp_list=poses.valid_robot_T_tcp_list,
image_list=poses.valid_image_list,
camera_T_target_list=poses.camera_T_target_list,
intrinsic_matrix=K,
distortion_coefficients=d,
)What tcp_T_camera Means
tcp_T_camera is the pose of the camera optical frame (OpenCV convention: x right, y down, z forward along the viewing direction) expressed in the frame your robot_T_tcp_list refers to, TCP or flange, whichever the controller reported. It maps camera-frame points into that frame:
p_tcp = result.tcp_T_camera @ p_camera # homogeneous (4,) points
robot_T_camera = robot_T_tcp @ result.tcp_T_cameraTranslation is in the same units as the robot poses and target, which Axon expects in meters.
Joint Refinement
The closed-form AX = XB solvers use one pose per view and treat every view as equally reliable, which biases them when the rotations between views are small. With refine=True (the default), calibrate() starts from the closed-form result and minimizes the reprojection error of every detected corner through the full chain base_T_target → robot_T_tcp → tcp_T_camera → camera, over tcp_T_camera and a single static board pose. A Huber loss down-weights the few corners that do not fit.
- When this call estimated the intrinsics (full pipeline),
Kand the distortion coefficients are refined jointly too. The refined camera model is only kept if no parameter moves more than 3 standard deviations of the image-only intrinsic fit. A larger move means robot pose error or a wrong board size is leaking intoK, so refinement runs again withKfixed andrefinement.notesays why. - With known intrinsics,
Kanddstay fixed. - With strict pass-through there are no corner detections, so no refinement runs (
has_refinementisFalse).
The refined values replace tcp_T_camera (and, for a joint refinement, intrinsic_matrix / distortion_coefficients) only when refinement.accepted is True. The closed-form solution is always kept in closed_form_tcp_T_camera. Pass refine=False to get the closed-form result alone.
Hand-Eye Methods
HAND_EYE_METHOD_BY_NAME_MAP maps method names to cv::HandEyeCalibrationMethod values:
| Method | Notes |
|---|---|
"TSAI" | Default; robust general-purpose choice. |
"PARK" | |
"HORAUD" | |
"ANDREFF" | |
"DANIILIDIS" | Occasionally diverges to an outlier solution, see Compare Hand-Eye Methods for cross-method comparison. |
With refinement on, the method only sets the starting point, so methods that start close to the answer end at the same refined result.
from telekinesis.axon.calibration import HAND_EYE_METHOD_BY_NAME_MAPHandEyeResult
| Field | Type | Description |
|---|---|---|
ok | bool | True only when the solver produced a valid rigid transform from at least 4 views. False when too few views survive, when every view has the same orientation (rotation is unobservable; rotate about at least two axes), or when the solver output is not a rigid transform. tcp_T_camera is then empty. |
tcp_T_camera | (4, 4) ndarray | Pose of the camera optical frame in the TCP (or flange) frame, see What tcp_T_camera Means. Refined when refinement.accepted. |
closed_form_tcp_T_camera | (4, 4) ndarray | The closed-form solver's transform, before refinement. |
intrinsic_matrix | (3, 3) ndarray | Camera matrix; jointly refined when the intrinsic stage ran and the joint refinement was accepted. Empty in the strict pass-through path when none was supplied. |
distortion_coefficients | ndarray | Distortion coefficients, may be empty, see above. |
reprojection_error | float | RMS reprojection error (px) of the intrinsic fit; NaN when no intrinsic stage ran (known intrinsics or strict pass-through). |
valid_robot_T_tcp_list | list[ndarray] | Robot poses used in the solve. |
valid_image_list | list[ndarray] | Images used in the solve (empty in strict pass-through when no matching images were passed). |
camera_T_target_list | list[(4, 4) ndarray] | Per-frame board poses used in the solve. |
has_consistency | bool | True when the solve succeeded and consistency is populated. |
consistency | TargetConsistencyStats | See Eye-in-Hand State. |
has_refinement | bool | True when joint refinement ran and refinement is populated. Check refinement.accepted to see whether it was used. |
refinement | HandEyeRefinement | See HandEyeRefinement below. |
valid_robot_T_tcp_list, valid_image_list and camera_T_target_list are aligned one-to-one: entry i of each describes the same frame. Views dropped by detection, by the per-view error filter, or by a failed board pose are removed from all three, so pass them together to tools such as hand_eye_roundtrip().
HandEyeRefinement
from telekinesis.axon.calibration import HandEyeRefinement| Field | Type | Description |
|---|---|---|
accepted | bool | True when the refined model is valid and did not increase the reprojection error. The fields below then hold refined values; otherwise they echo the starting values. |
converged | bool | True when the optimizer stopped on its tolerances rather than the iteration cap. |
iterations | int | Optimizer iterations used. |
rms_before_px | float | RMS corner reprojection error (px) through the full robot-camera chain, starting from the closed-form result. |
rms_after_px | float | The same, after refinement. |
refined_intrinsics | bool | Whether K and distortion were part of the optimization. |
tcp_T_camera | (4, 4) ndarray | Refined hand-eye transform. |
base_T_target | (4, 4) ndarray | Refined pose of the static board in the robot base frame. |
intrinsic_matrix | (3, 3) ndarray | Refined (or passed-through) camera matrix. |
distortion_coefficients | ndarray | Refined (or passed-through) distortion coefficients. |
note | str | Explanation of non-default outcomes, e.g. why joint intrinsic refinement was not used. Empty otherwise. |
READING THE REFINEMENT
rms_after_px measures how well one fixed board, one tcp_T_camera and your robot poses explain every corner at once. If it is much larger than the intrinsic reprojection_error, suspect the robot poses, the board size, or a board that moved, rather than the camera. base_T_target also gives you the board's location in the robot base frame.
See EyeInHandCalibrator for theory and best practices, and EyeInHandCalibrator State to read back the last result.