Skip to content

Calibrate ​

SUMMARY

calibrate() solves tcp_T_camera, the fixed transform between a robot's TCP (flange) and a wrist-mounted camera, from images of a static calibration board taken at a series of robot poses. A closed-form hand-eye solve is followed by a joint refinement over every detected corner.

SUPPORTED TARGETS

Works with any target, ChessboardTarget, CharucoTarget, ArucoTarget, or ZividTarget, see All Supported Targets. CharucoTarget is recommended, since it gives a pose from partial views.

UNITS

Input images can be BGR, BGRA or grayscale ndarrays. robot_T_tcp_list and tcp_T_camera are (4, 4) SE(3) matrices with translation in meters. reprojection_error and the refinement RMS values are in pixels.

The Skill ​

python
result = calibrator.calibrate(
    robot_T_tcp_list,
    image_list,
    method="TSAI",
    camera_T_target_list=[],
    intrinsic_matrix=None,
    distortion_coefficients=None,
    output_path=None,
    refine=True,
)
ParameterTypeDescription
robot_T_tcp_listlist[(4, 4) ndarray]Per-frame robot base to TCP (or flange) transforms, aligned one-to-one with image_list.
image_listlist[ndarray]Per-frame images of the static board.
methodstrClosed-form hand-eye method, see Hand-Eye Methods. Default "TSAI".
camera_T_target_listlist[(4, 4) ndarray]Optional pre-computed board poses, see the input modes below.
intrinsic_matrix(3, 3) ndarrayOptional known camera matrix.
distortion_coefficientsndarrayOptional known distortion coefficients.
output_pathstr | Path | NoneWhen set, writes a hand_eye.json result file (directory or file path), see Hand-Eye Result. Default None writes nothing.
refineboolRun joint reprojection refinement after the closed-form solve, see Joint Refinement. Default True.

There are three input modes, selected by which optional arguments are supplied:

ModeHow to triggerBehavior
Full pipeline (default)Leave intrinsic_matrix and camera_T_target_list unsetRuns intrinsic calibration on image_list, estimates a board pose on each view kept by the intrinsic fit, solves cv::calibrateHandEye, then refines.
Known intrinsicsPass intrinsic_matrix (and optionally distortion_coefficients)Skips the intrinsic stage; per-frame board poses come from target detection + planar PnP against the given K, d. Refinement runs with K fixed.
Strict pass-throughPass camera_T_target_listSkips detection entirely. robot_T_tcp_list must already be aligned to the supplied poses. No refinement, since there are no corner detections.

calibrate() raises ValueError for an unknown method, when image_list and robot_T_tcp_list (or camera_T_target_list) have different lengths, and when an input pose is not a valid rigid 4x4 transform (orthonormal rotation, det = +1, bottom row [0, 0, 0, 1]).

The Code ​

Requires images of a calibration board alongside the robot's TCP pose (robot_T_tcp, as 4x4 homogeneous matrices) for each frame.

python
from telekinesis.axon import EyeInHandCalibrator
from telekinesis.axon.targets import CharucoTarget

target = CharucoTarget(squares_x=6, squares_y=9, square_length=0.012, marker_length=0.009)

calibrator = EyeInHandCalibrator(target)

result = calibrator.calibrate(
    robot_T_tcp_list=robot_poses,  # list of (4, 4) np.ndarray
    image_list=images,
    method="TSAI",
)

result.ok                        # bool
result.tcp_T_camera              # (4, 4) np.ndarray, refined when refinement was accepted
result.closed_form_tcp_T_camera  # (4, 4) np.ndarray, before refinement
result.intrinsic_matrix
result.distortion_coefficients
result.reprojection_error        # intrinsic RMS (px)
result.refinement.rms_after_px   # corner RMS through the whole robot-camera chain, when result.has_refinement
result.consistency               # TargetConsistencyStats, when result.has_consistency

Use known intrinsics ​

If the camera is already calibrated, pass its intrinsics. Board poses are then computed per frame with that K, for any target type:

python
result = calibrator.calibrate(
    robot_T_tcp_list=robot_poses,
    image_list=images,
    intrinsic_matrix=K,
    distortion_coefficients=d,
)

Pass pre-computed per-frame poses ​

If you have camera_T_target transforms from an external source, pass them via camera_T_target_list to skip detection and pose estimation (see compute_camera_T_target_list() for how to produce them). robot_T_tcp_list must be filtered to the same frames:

python
poses = calibrator.compute_camera_T_target_list(images, robot_poses, K, d)

result = calibrator.calibrate(
    robot_T_tcp_list=poses.valid_robot_T_tcp_list,
    image_list=poses.valid_image_list,
    camera_T_target_list=poses.camera_T_target_list,
    intrinsic_matrix=K,
    distortion_coefficients=d,
)

What tcp_T_camera Means ​

tcp_T_camera is the pose of the camera optical frame (OpenCV convention: x right, y down, z forward along the viewing direction) expressed in the frame your robot_T_tcp_list refers to, TCP or flange, whichever the controller reported. It maps camera-frame points into that frame:

python
p_tcp = result.tcp_T_camera @ p_camera      # homogeneous (4,) points
robot_T_camera = robot_T_tcp @ result.tcp_T_camera

Translation is in the same units as the robot poses and target, which Axon expects in meters.

Joint Refinement ​

The closed-form AX = XB solvers use one pose per view and treat every view as equally reliable, which biases them when the rotations between views are small. With refine=True (the default), calibrate() starts from the closed-form result and minimizes the reprojection error of every detected corner through the full chain base_T_target → robot_T_tcp → tcp_T_camera → camera, over tcp_T_camera and a single static board pose. A Huber loss down-weights the few corners that do not fit.

  • When this call estimated the intrinsics (full pipeline), K and the distortion coefficients are refined jointly too. The refined camera model is only kept if no parameter moves more than 3 standard deviations of the image-only intrinsic fit. A larger move means robot pose error or a wrong board size is leaking into K, so refinement runs again with K fixed and refinement.note says why.
  • With known intrinsics, K and d stay fixed.
  • With strict pass-through there are no corner detections, so no refinement runs (has_refinement is False).

The refined values replace tcp_T_camera (and, for a joint refinement, intrinsic_matrix / distortion_coefficients) only when refinement.accepted is True. The closed-form solution is always kept in closed_form_tcp_T_camera. Pass refine=False to get the closed-form result alone.

Hand-Eye Methods ​

HAND_EYE_METHOD_BY_NAME_MAP maps method names to cv::HandEyeCalibrationMethod values:

MethodNotes
"TSAI"Default; robust general-purpose choice.
"PARK"
"HORAUD"
"ANDREFF"
"DANIILIDIS"Occasionally diverges to an outlier solution, see Compare Hand-Eye Methods for cross-method comparison.

With refinement on, the method only sets the starting point, so methods that start close to the answer end at the same refined result.

python
from telekinesis.axon.calibration import HAND_EYE_METHOD_BY_NAME_MAP

HandEyeResult ​

FieldTypeDescription
okboolTrue only when the solver produced a valid rigid transform from at least 4 views. False when too few views survive, when every view has the same orientation (rotation is unobservable; rotate about at least two axes), or when the solver output is not a rigid transform. tcp_T_camera is then empty.
tcp_T_camera(4, 4) ndarrayPose of the camera optical frame in the TCP (or flange) frame, see What tcp_T_camera Means. Refined when refinement.accepted.
closed_form_tcp_T_camera(4, 4) ndarrayThe closed-form solver's transform, before refinement.
intrinsic_matrix(3, 3) ndarrayCamera matrix; jointly refined when the intrinsic stage ran and the joint refinement was accepted. Empty in the strict pass-through path when none was supplied.
distortion_coefficientsndarrayDistortion coefficients, may be empty, see above.
reprojection_errorfloatRMS reprojection error (px) of the intrinsic fit; NaN when no intrinsic stage ran (known intrinsics or strict pass-through).
valid_robot_T_tcp_listlist[ndarray]Robot poses used in the solve.
valid_image_listlist[ndarray]Images used in the solve (empty in strict pass-through when no matching images were passed).
camera_T_target_listlist[(4, 4) ndarray]Per-frame board poses used in the solve.
has_consistencyboolTrue when the solve succeeded and consistency is populated.
consistencyTargetConsistencyStatsSee Eye-in-Hand State.
has_refinementboolTrue when joint refinement ran and refinement is populated. Check refinement.accepted to see whether it was used.
refinementHandEyeRefinementSee HandEyeRefinement below.

valid_robot_T_tcp_list, valid_image_list and camera_T_target_list are aligned one-to-one: entry i of each describes the same frame. Views dropped by detection, by the per-view error filter, or by a failed board pose are removed from all three, so pass them together to tools such as hand_eye_roundtrip().

HandEyeRefinement ​

python
from telekinesis.axon.calibration import HandEyeRefinement
FieldTypeDescription
acceptedboolTrue when the refined model is valid and did not increase the reprojection error. The fields below then hold refined values; otherwise they echo the starting values.
convergedboolTrue when the optimizer stopped on its tolerances rather than the iteration cap.
iterationsintOptimizer iterations used.
rms_before_pxfloatRMS corner reprojection error (px) through the full robot-camera chain, starting from the closed-form result.
rms_after_pxfloatThe same, after refinement.
refined_intrinsicsboolWhether K and distortion were part of the optimization.
tcp_T_camera(4, 4) ndarrayRefined hand-eye transform.
base_T_target(4, 4) ndarrayRefined pose of the static board in the robot base frame.
intrinsic_matrix(3, 3) ndarrayRefined (or passed-through) camera matrix.
distortion_coefficientsndarrayRefined (or passed-through) distortion coefficients.
notestrExplanation of non-default outcomes, e.g. why joint intrinsic refinement was not used. Empty otherwise.

READING THE REFINEMENT

rms_after_px measures how well one fixed board, one tcp_T_camera and your robot poses explain every corner at once. If it is much larger than the intrinsic reprojection_error, suspect the robot poses, the board size, or a board that moved, rather than the camera. base_T_target also gives you the board's location in the robot base frame.

See EyeInHandCalibrator for theory and best practices, and EyeInHandCalibrator State to read back the last result.