MultiCameraCalibrator

SUMMARY

MultiCameraCalibrator solves the pose of every camera in an N >= 2 synchronized rig relative to a reference camera, via pairwise cv::stereoCalibrate, then validates the result by triangulating the calibration board back into 3D.

The Skill

from telekinesis.axon import MultiCameraCalibrator

The Code

from telekinesis.axon import MultiCameraCalibrator
from telekinesis.axon.targets import CharucoTarget

target = CharucoTarget(squares_x=6, squares_y=9, square_length=0.012, marker_length=0.009)

calibrator = MultiCameraCalibrator(target, num_cameras=3, reference_index=1)
result = calibrator.calibrate(image_lists)  # image_lists[cam][frame]

result.ok                          # True iff every non-reference pair solved
result.reference_T_camera_list     # per-camera (4, 4) np.ndarray
result.intrinsic_matrices
result.pair_reprojection_errors

Each non-reference camera is paired against the reference camera; returned transforms express each camera's pose in the reference camera's frame (reference_T_camera_i). Inputs are per-camera image lists captured synchronously, frame k across all cameras must show the same target pose.

Theory

MultiCameraCalibrator never solves for every camera pair at once. Instead, it designates one camera as the reference and runs cv::stereoCalibrate between the reference and each other camera independently, using the same synchronized frames of the shared target. Each solve produces reference_T_camera_i directly, there's no chain of transforms to compose, since every non-reference camera is related to the reference by exactly one stereo pair.

calibrate() reuses each camera's own intrinsic calibration (CALIB_FIX_INTRINSIC, the default stereo_calibrate_flags) rather than re-deriving K and distortion from the stereo pairs themselves, stereo-only intrinsics are typically less well constrained than a dedicated single-camera calibration with full coverage of the frame (see IntrinsicCalibrator Best Practices).

Because every reference_T_camera_i comes from an independent 2D reprojection fit, a low pair_reprojection_errors value doesn't by itself prove the rig's 3D geometry is self-consistent. triangulate() provides that independent check: it reconstructs the target's ChArUco corners in 3D from the already-solved camera geometry and compares the result against the board's known physical layout (see Best Practices), a form of validation the 2D reprojection fit alone can't provide.

Initialization

MultiCameraCalibrator(
    target,
    num_cameras,
    reference_index=0,
    options=MultiCameraCalibrator.default_options(),
    stereo_calibrate_flags=CALIB_FIX_INTRINSIC,
)

Parameter	Type	Description
target	ChessboardTarget | CharucoTarget | ArucoTarget	Calibration target shared by all cameras, see All Supported Targets.
num_cameras	int	Number of cameras (>= 2).
reference_index	int	Index of the reference camera. Default 0.
options	IntrinsicOptions	Intrinsic-stage tunables, see IntrinsicOptions. Defaults to default_options() below, since multi-camera rigs typically need a more relaxed per_view_error_threshold than a single well-lit intrinsic calibration.
stereo_calibrate_flags	int	cv::CALIB_* bitmask for stereoCalibrate. Default CALIB_FIX_INTRINSIC.

Raises on invalid camera counts or reference indices.

Static Method	Returns	Description
default_options()	IntrinsicOptions	per_view_error_threshold=3.0 with stereo-grade termination criteria.

Skills

Skill	Description
Calibrate	Solve the pose of every camera in a synchronized rig relative to a reference camera.
Triangulate	Reconstruct the target's ChArUco corners in 3D and validate against known board geometry.
Calibrate Subsets	Calibrate and triangulate every camera subset of a given size.

See also MultiCameraCalibrator State to read back the last result, camera count, reference index, and a formatted report.

Best Practices

• Maximize FOV overlap with the reference camera. Every non-reference camera is solved purely against the reference, a camera with little shared view of the target's positions will get few usable frames for its stereo pair, regardless of how good the rest of the rig is.
• Pick the most centrally-overlapping camera as the reference, not just camera 0. If one camera sees the target across the widest range of positions the others also see, that camera minimizes dropped frames across all pairs.
• Move the target through the region every camera shares, not just the reference's own field of view, frames where the target is only visible to a subset of cameras are wasted for the pairs that can't see it. Keep at least kMinSharedCornersPerFrame (15) corners in common between a pair for a frame to count.
• Check pair_reprojection_errors per camera, not just whether ok is True, ok only means every pair produced a transform, not that every transform is equally good. A camera with a much higher pair error than the rest is usually a sign of poor overlap or a synchronization issue with that specific camera.
• Use triangulate() as a hard cross-check, not just calibrate()'s own errors. Compare neighbor_distances_m against the board's actual printed geometry and check plane_thickness_m is small, a triangulated board that isn't flat or doesn't match its known dimensions means the extrinsics are wrong even if the 2D reprojection errors looked fine.
• Run calibrate_subsets() when a rig underperforms to isolate which camera (or pair) is dragging down the result, rather than re-capturing the whole dataset blind.