Run Benchmark

SUMMARY

run() benchmarks one target/IntrinsicOptions configuration: cross-validated reprojection error, plus an optional hand-eye round-trip error. Call it once per configuration you want to compare, and collect the BenchmarkResults yourself. See CalibrationBenchmark for the class this skill belongs to.

SUPPORTED TARGETS

The cross-validation metric works with any target, ChessboardTarget, CharucoTarget, or ArucoTarget. The optional hand-eye round-trip metric (robot_poses + eye_in_hand_options) requires a CharucoTarget.

UNITS

Input images are BGR ndarrays and robot_poses are (4, 4) SE(3) matrices with translation in meters. Both held_out_reprojection_error and hand_eye_roundtrip_error are in pixels (-1 when not computed).

The Skill

result = bench.run(
    images=my_images,
    target=target,
    options=IntrinsicOptions(),
    robot_poses=my_robot_poses,             # optional, enables the round-trip metric
    eye_in_hand_options=IntrinsicOptions(), # optional
    n_splits=5,
)

Skill	Returns	Description
run(images, target, options, robot_poses=[], eye_in_hand_options=None, backend_name="OpenCV", n_splits=5, seed=0)	BenchmarkResult	Full benchmark: held_out_reprojection_error via cross-validation, plus hand_eye_roundtrip_error when robot_poses and eye_in_hand_options are both set.

Parameter	Type	Description
images	list[ndarray]	BGR calibration images.
target	Target	Calibration target to detect.
options	IntrinsicOptions	Intrinsic solve configuration under test, see IntrinsicOptions.
robot_poses	list[(4, 4) ndarray]	Optional per-frame robot poses. Set together with eye_in_hand_options to enable the round-trip metric.
eye_in_hand_options	IntrinsicOptions | None	Optional options for the eye-in-hand stage.
backend_name	str	Label recorded on the result, e.g. "OpenCV".
n_splits	int	Cross-validation fold count. Default 5.
seed	int	Makes the fold shuffle deterministic and reproducible across runs. Default 0.

The Code

from telekinesis.axon import CalibrationBenchmark, IntrinsicOptions
from telekinesis.axon.targets import CharucoTarget

target = CharucoTarget(squares_x=6, squares_y=9, square_length=0.012, marker_length=0.009)

bench = CalibrationBenchmark()
result = bench.run(
    images=my_images,
    target=target,
    options=IntrinsicOptions(),
    robot_poses=my_robot_poses,             # optional, enables the round-trip metric
    eye_in_hand_options=IntrinsicOptions(), # optional
    n_splits=5,
)
CalibrationBenchmark.save_json([result], "calibration_benchmark.json")
print(CalibrationBenchmark.format_results([result]))

Output (calibration_benchmark.json):

{
  "run_timestamp": "2026-05-18T...",
  "results": [
    {
      "backend_name": "OpenCV",
      "reprojection_error": 0.42,
      "held_out_reprojection_error": 0.61,
      "hand_eye_roundtrip_error": 1.3,
      "camera_matrix": [[fx, 0, cx], [0, fy, cy], [0, 0, 1]],
      "dist_coeffs": [...]
    }
  ]
}

BenchmarkResult

Field	Type	Description
backend_name	str	Label for this configuration, e.g. "OpenCV".
reprojection_error	float	Full-fit reprojection error over all images.
held_out_reprojection_error	float	Mean cross-validation error over held-out folds.
hand_eye_roundtrip_error	float	Round-trip error; -1 when not computed.
camera_matrix	(3, 3) ndarray	Camera matrix from the full fit.
dist_coeffs	ndarray	Distortion coefficients from the full fit.
timestamp	str	ISO-8601 UTC timestamp of the run.
image_count	int	Number of images used.