Register Point Clouds Using Point to Plane ICP

SUMMARY

Register Point Clouds Using Point to Plane ICP aligns two point clouds by minimizing each source point's distance to the tangent plane at its matched target point, using the target's surface normals.

Like register_point_clouds_using_point_to_point_icp, but instead of measuring correspondence error as raw point-to-point distance, it measures the distance from a source point to the tangent plane at its matched target point. This typically converges faster and more accurately when normals are available, especially on smooth surfaces. It requires source_point_cloud/target_point_cloud to already be roughly aligned, within max_correspondence_distance — use a coarse method first (the SDK's register_point_clouds_using_centroid_translation, register_point_clouds_using_cuboid_translation_sampler_icp, or register_point_clouds_using_fast_global_registration) if they aren't. The function returns the 4x4 transform found, not an already-moved point cloud — apply it with apply_transform_to_point_cloud.

Use this Skill when you want to refine an already roughly-aligned point cloud pair to a high-accuracy final alignment, using surface normals.

The Skill

from telekinesis import vitreous
import numpy as np

transformation_matrix = vitreous.register_point_clouds_using_point_to_plane_icp(
    source_point_cloud=source_point_cloud,
    target_point_cloud=target_point_cloud,
    initial_transformation_matrix=np.eye(4),
    max_iterations=500,
    max_correspondence_distance=30,
    normal_max_neighbors=20,
    normal_search_radius=2,
    use_robust_kernel=False,
    loss_type="tukey_loss",
    noise_standard_deviation=10,
)

aligned_point_cloud = vitreous.apply_transform_to_point_cloud(
    point_cloud=source_point_cloud,
    transformation_matrix=transformation_matrix,
)

API Reference

Full parameter and return type documentation for register_point_clouds_using_point_to_plane_icp.

View Reference →

Data Transfer Notice

There is no longer a fixed limit of 1 million points per request. However, very large datasets may result in slower data transfer and processing times. We are continuously optimizing performance as part of our beta program, with ongoing improvements to enhance speed and reliability.

Example

Source and Target Point Clouds

Raw sensor input i.e. target point cloud in green and object model in red

Registered Point Clouds

Registered source point cloud (red) aligned to target point cloud (green) using point-to-plane ICP

The Code

"""
Demonstrates aligning point clouds using Point-to-Plane ICP.
"""

import numpy as np
from loguru import logger
import rerun as rr

from telekinesis import vitreous, datatypes


def register_point_clouds_using_point_to_plane_icp_example():
    """
    Aligns point clouds using Point-to-Plane ICP.

    Minimizes point-to-tangent-plane distances instead of point-to-point. More
    accurate than point-to-point ICP, especially for planar surfaces.
    """
    # ===================== Load Data ==========================================
    source_point_cloud_url = "https://assets.telekinesis.ai/examples/v1/point_clouds/weld_clamp_model_shifted.ply"
    target_point_cloud_url = "https://assets.telekinesis.ai/examples/v1/point_clouds/weld_clamp_cluster_0_centroid_registered.ply"
    source_point_cloud = datatypes.PointCloud.from_url(
        url=source_point_cloud_url, use_cache=True
    )
    target_point_cloud = datatypes.PointCloud.from_url(
        url=target_point_cloud_url, use_cache=True
    )

    # ===================== Run Skill ==========================================
    transformation_matrix = vitreous.register_point_clouds_using_point_to_plane_icp(
        max_iterations=500,
        max_correspondence_distance=30,
        normal_max_neighbors=20,
        normal_search_radius=2,
        use_robust_kernel=False,
        loss_type="tukey_loss",
        noise_standard_deviation=10,
        source_point_cloud=source_point_cloud,
        target_point_cloud=target_point_cloud,
        initial_transformation_matrix=np.eye(4),
    )

    # ===================== Log ================================================
    logger.success(
        f"Registered {source_point_cloud} to {target_point_cloud} using point-to-plane ICP"
    )
    logger.success(f"Results: {transformation_matrix}")
    logger.info(f"Transformation matrix data: {transformation_matrix.data}")
    logger.info(f"Transformation matrix shape: {transformation_matrix.shape}")
    logger.info(f"Transformation matrix ndim: {transformation_matrix.ndim}")
    logger.info(f"Transformation matrix dtype: {transformation_matrix.dtype}")

    # ===================== Visualization  (Optional) ===========================
    aligned_source_point_cloud = vitreous.apply_transform_to_point_cloud(
        point_cloud=source_point_cloud,
        transformation_matrix=transformation_matrix,
    )

    rr.init("register_point_clouds_using_point_to_plane_icp_example", spawn=True)
    datatypes.visualize(source_point_cloud, entity_path="/1-before_registration_source")
    datatypes.visualize(target_point_cloud, entity_path="/2-before_registration_target")
    datatypes.visualize(
        aligned_source_point_cloud, entity_path="/3-after_registration_source_aligned"
    )


if __name__ == "__main__":
    register_point_clouds_using_point_to_plane_icp_example()

Runnable examples are available in the Telekinesis examples repository.

Follow the README in that repository to set up the environment, run this specific example with:

cd telekinesis-examples
python examples/point_cloud/register_point_clouds_using_point_to_plane_icp.py

Parameter Configuration

Parameter	Type	Default	Description
source_point_cloud	datatypes.PointCloud	required	The point cloud to align. Normals are estimated automatically if not already present
target_point_cloud	datatypes.PointCloud	required	The point cloud to align to. Normals are estimated automatically if not already present
initial_transformation_matrix	datatypes.Mat4x4 | np.ndarray | list[list[float]]	np.eye(4)	A 4x4 transform applied to source_point_cloud before ICP starts — should already bring it within max_correspondence_distance of target_point_cloud
max_iterations	datatypes.Int | int	50	Maximum number of ICP iterations to run. Must be > 0
max_correspondence_distance	datatypes.Float | float | int	50	Maximum distance, in the point clouds' coordinate units, at which two points are considered a match. Must be > 0
normal_max_neighbors	datatypes.Int | int	30	Maximum number of neighbors used for normal estimation (when normals aren't already present). Must be > 0
normal_search_radius	datatypes.Float | float | int	0.05	Search radius, in the point clouds' coordinate units, used for normal estimation. Must be > 0
use_robust_kernel	datatypes.Bool | bool	False	Whether to down-weight outlier correspondences using the robust loss specified by loss_type, instead of the standard L2 loss
loss_type	datatypes.String | str	"tukey_loss"	The robust loss function used when use_robust_kernel is True (ignored otherwise). "tukey_loss" is the only currently functional value — "cauchy_loss", "huber_loss", or any other value cause the request to fail whenever use_robust_kernel is True
noise_standard_deviation	datatypes.Float | float | int	0.0	Expected standard deviation of point noise, in the point clouds' coordinate units, used to weight correspondences by confidence. Must be >= 0; 0 disables this weighting

Returns

Type	Description
datatypes.Mat4x4	The 4x4 transform found by ICP — this is the transform itself, not an already-transformed point cloud. Pass it to apply_transform_to_point_cloud (as transformation_matrix) to actually move source_point_cloud's points. Use .data for the raw (4, 4) array. If the alignment's fitness falls below an internal threshold, this silently returns the identity transform instead of raising.

Raises

Exception	Condition
TypeError	A parameter's value does not match its expected type (see the Parameter Configuration table above)
ValueError	initial_transformation_matrix is not shape (4, 4) (or, for a list input, contains a non-numeric element); or max_iterations, max_correspondence_distance, normal_max_neighbors, or normal_search_radius is not positive
ConfigurationError	The TELEKINESIS_API_KEY environment variable is not set
SerializationError	The request input failed to serialize, the response was not returned as an Arrow stream, or the response failed to deserialize
RequestTimeoutError	The request to the Vitreous service timed out
TransportError	A network failure occurred before a response was received
ClientError	The Vitreous service rejected the request due to invalid or malformed input (HTTP 400/422), an unrecognized endpoint (HTTP 404), or another unexpected 4xx response
AuthenticationError	The API key was rejected as invalid or expired (HTTP 401)
AuthenticationServiceError	The authentication service returned an invalid response, was temporarily unavailable, or timed out (HTTP 502/503/504)
ServerError	The Vitreous service returned a 5xx or otherwise unexpected error response

How to Tune the Parameters

The register_point_clouds_using_point_to_plane_icp Skill exposes eight tunable parameters that control the ICP iteration budget, matching distance, normal estimation, and outlier robustness.

initial_transformation_matrix

• Controls: The starting transform applied to source_point_cloud before ICP begins.
• Default: np.eye(4) (identity)
• Must already bring source_point_cloud within max_correspondence_distance of target_point_cloud — this Skill does not search for a rough alignment itself.

max_iterations

• Controls: The maximum number of ICP iterations to run.
• Units: Iterations (integer)
• Default: 50
• Must be > 0
• Increase → more refinement, but slower
• Decrease → faster, but may stop before fully converging
• Typical range: 10-200 — use 10-30 for fast, 30-50 for balanced, 50-200 for high precision

max_correspondence_distance

• Controls: The maximum distance, in the point clouds' coordinate units, at which two points are considered a match.
• Units: The point clouds' coordinate units (e.g. meters or millimeters, whatever unit the input data uses)
• Default: 50
• Must be > 0
• Increase → allows matching more distant points, risking incorrect matches
• Decrease → requires closer matches
• Set to roughly 2-5x the point spacing
• Typical range: scale relative to your point spacing and coordinate units — the default of 50 assumes a millimeter-or-larger-scale point cloud; rescale down (e.g. to 0.01-0.1) for a meter-scale point cloud

normal_max_neighbors

• Controls: The maximum number of neighbors used for normal estimation, when normals aren't already present on the point clouds.
• Units: Points (integer count)
• Default: 30
• Must be > 0
• Increase → smoother, more stable normals, but slower
• Decrease → faster
• Typical range: 10-50

normal_search_radius

• Controls: The search radius, in the point clouds' coordinate units, used for normal estimation.
• Units: The point clouds' coordinate units
• Default: 0.05
• Must be > 0
• Increase → considers more neighbors
• Decrease → faster
• Set to roughly 2-5x the point spacing
• Typical range: 0.01-0.1 (in coordinate units matching a meter-scale point cloud)

use_robust_kernel

• Controls: Whether correspondence errors are down-weighted using the robust loss in loss_type, instead of the standard L2 loss.
• Default: False
• Set to True when the point clouds contain outliers or noisy points that could otherwise skew the alignment.

loss_type

• Controls: The robust loss function used to weight correspondence errors when use_robust_kernel is True (ignored otherwise).
• Default: "tukey_loss"
• "tukey_loss" is the only currently functional robust loss. "cauchy_loss" and "huber_loss" are recognized names but are not implemented — using either (or any other value) while use_robust_kernel=True causes the request to fail. Since loss_type is ignored when use_robust_kernel=False (the default), you generally don't need to touch this parameter unless you're also turning on use_robust_kernel.

noise_standard_deviation

• Controls: The expected standard deviation of point noise, in the point clouds' coordinate units, used to weight correspondences by confidence.
• Units: The point clouds' coordinate units
• Default: 0.0
• Must be >= 0; 0 disables this weighting
• Increase → gives less weight to noisier correspondences
• Typical range: 0.0 to disable, or a small fraction of your point spacing for typical sensor noise

TIP

Point-to-plane ICP needs reliable normals to outperform point-to-point ICP — if normal_max_neighbors/normal_search_radius are too small for your point spacing, the estimated normals will be noisy and convergence quality will suffer. When your data has outliers, turn on use_robust_kernel before reaching for a smaller max_correspondence_distance.

Where to Use the Skill

Common pipelines include:

• Fine alignment refinement – tightening a coarse registration (from register_point_clouds_using_fast_global_registration or a sampler-based ICP Skill) into a precise final transform
• Precise 6D pose estimation – computing an accurate object pose for robotic grasping once a rough alignment is available
• Surface-based registration – aligning scans of smooth or planar parts where normals are reliable and point-to-point distance alone converges poorly
• Quality inspection – measuring fine deviations between a scanned part and its reference model after alignment

Alternative Skills

Skill	vs. Register Point Clouds Using Point to Plane ICP
register_point_clouds_using_point_to_point_icp	Matches by raw point-to-point distance instead of distance-to-tangent-plane. Use it when normals are unavailable or unreliable; use point-to-plane ICP when normals are available, since it typically converges faster and more accurately.
register_point_clouds_using_fast_global_registration	Feature-based registration that doesn't require rough pre-alignment. Run it first if the clouds aren't already roughly positioned within max_correspondence_distance.
register_point_clouds_using_cuboid_translation_sampler_icp	Grid-searches translations with repeated ICP runs to establish a rough alignment. An alternative coarse pre-alignment step when only the translation is uncertain.
register_point_clouds_using_rotation_sampler_icp	Grid-searches rotations with repeated ICP runs. An alternative coarse pre-alignment step when only the rotation is uncertain.

register_point_clouds_using_centroid_translation is the fastest coarse pre-alignment option among the ones this Skill's docstring recommends, useful when only a translation offset (not a rotation) separates the clouds.

When Not to Use the Skill

Do not use Register Point Clouds Using Point to Plane ICP when:

• The point clouds aren't already roughly aligned within max_correspondence_distance – run a coarse method first, such as register_point_clouds_using_centroid_translation, register_point_clouds_using_cuboid_translation_sampler_icp, register_point_clouds_using_rotation_sampler_icp, or register_point_clouds_using_fast_global_registration
• Normals are unavailable or unreliable (e.g. very sparse or noisy point clouds) – use register_point_clouds_using_point_to_point_icp instead
• The surfaces are not smooth or well-defined – the tangent-plane metric works best on smooth surfaces; use point-to-point ICP for irregular or highly detailed geometry
• You need the fastest possible registration – point-to-plane ICP does extra work (normal estimation and tangent-plane distances) compared to point-to-point ICP

TIP

If you're unsure whether your point clouds already have normals, you don't need to compute them yourself — this Skill estimates normals automatically using normal_max_neighbors/normal_search_radius whenever they're missing.