Register Point Clouds Using Point to Plane ICP

SUMMARY

Register Point Clouds Using Point to Plane ICP aligns two point clouds by minimizing each source point's distance to the tangent plane at its matched target point, using the target's surface normals.

Like register_point_clouds_using_point_to_point_icp, but instead of measuring correspondence error as raw point-to-point distance, it measures the distance from a source point to the tangent plane at its matched target point. This typically converges faster and more accurately when normals are available, especially on smooth surfaces. It requires source_point_cloud/target_point_cloud to already be roughly aligned, within max_correspondence_distance — use a coarse method first (the SDK's register_point_clouds_using_centroid_translation, register_point_clouds_using_cuboid_translation_sampler_icp, or register_point_clouds_using_fast_global_registration) if they aren't. The function returns the 4x4 transform found, not an already-moved point cloud — apply it with apply_transform_to_point_cloud.

Use this Skill when you want to refine an already roughly-aligned point cloud pair to a high-accuracy final alignment, using surface normals.

The Skill

from telekinesis import vitreous
import numpy as np

transformation_matrix = vitreous.register_point_clouds_using_point_to_plane_icp(
    source_point_cloud=source_point_cloud,
    target_point_cloud=target_point_cloud,
    initial_transformation_matrix=np.eye(4),
    max_iterations=500,
    max_correspondence_distance=30,
    normal_max_neighbors=20,
    normal_search_radius=2,
    use_robust_kernel=False,
    loss_type="tukey_loss",
    noise_standard_deviation=10,
)

aligned_point_cloud = vitreous.apply_transform_to_point_cloud(
    point_cloud=source_point_cloud,
    transformation_matrix=transformation_matrix,
    modify_inplace=False,
)

API Reference

Full parameter and return type documentation for register_point_clouds_using_point_to_plane_icp.

View Reference →

Data Transfer Notice

There is no longer a fixed limit of 1 million points per request. However, very large datasets may result in slower data transfer and processing times. We are continuously optimizing performance as part of our beta program, with ongoing improvements to enhance speed and reliability.

Example

Source and Target Point Clouds

Raw sensor input i.e. target point cloud in green and object model in red

Registered Point Clouds

Registered source point cloud (red) aligned to target point cloud (green) using point-to-plane ICP

The Code

"""
Demonstrates aligning point clouds using Point-to-Plane ICP.
"""

import numpy as np
from loguru import logger
import rerun as rr

from telekinesis import vitreous, datatypes


def register_point_clouds_using_point_to_plane_icp_example():
    """
    Aligns point clouds using Point-to-Plane ICP.

    Minimizes point-to-tangent-plane distances instead of point-to-point. More
    accurate than point-to-point ICP, especially for planar surfaces.
    """
    # ===================== Load Data ==========================================
    source_point_cloud_url = "https://assets.telekinesis.ai/examples/v1/point_clouds/weld_clamp_model_shifted.ply"
    target_point_cloud_url = "https://assets.telekinesis.ai/examples/v1/point_clouds/weld_clamp_cluster_0_centroid_registered.ply"
    source_point_cloud = datatypes.PointCloud.from_url(
        url=source_point_cloud_url, use_cache=True
    )
    target_point_cloud = datatypes.PointCloud.from_url(
        url=target_point_cloud_url, use_cache=True
    )

    # ===================== Run Skill ==========================================
    transformation_matrix = vitreous.register_point_clouds_using_point_to_plane_icp(
        max_iterations=500,
        max_correspondence_distance=30,
        normal_max_neighbors=20,
        normal_search_radius=2,
        use_robust_kernel=False,
        loss_type="tukey_loss",
        noise_standard_deviation=10,
        source_point_cloud=source_point_cloud,
        target_point_cloud=target_point_cloud,
        initial_transformation_matrix=np.eye(4),
    )

    # ===================== Log ================================================
    logger.success(
        f"Registered {source_point_cloud} to {target_point_cloud} using point-to-plane ICP"
    )
    logger.success(f"Results: {transformation_matrix}")
    logger.info(f"Transformation matrix data: {transformation_matrix.data}")
    logger.info(f"Transformation matrix shape: {transformation_matrix.shape}")
    logger.info(f"Transformation matrix ndim: {transformation_matrix.ndim}")
    logger.info(f"Transformation matrix dtype: {transformation_matrix.dtype}")

    # ===================== Visualization  (Optional) ===========================
    aligned_source_point_cloud = vitreous.apply_transform_to_point_cloud(
        point_cloud=source_point_cloud,
        transformation_matrix=transformation_matrix,
        modify_inplace=False,
    )

    rr.init("register_point_clouds_using_point_to_plane_icp_example", spawn=True)
    datatypes.visualize(source_point_cloud, entity_path="/1-before_registration_source")
    datatypes.visualize(target_point_cloud, entity_path="/2-before_registration_target")
    datatypes.visualize(
        aligned_source_point_cloud, entity_path="/3-after_registration_source_aligned"
    )


if __name__ == "__main__":
    register_point_clouds_using_point_to_plane_icp_example()

Runnable examples are available in the Telekinesis examples repository.

Follow the README in that repository to set up the environment, run this specific example with:

cd telekinesis-examples
python examples/point_cloud/register_point_clouds_using_point_to_plane_icp.py

Parameter Configuration

Key	Type	Default	Description
source_point_cloud	datatypes.PointCloud	required	The point cloud to align. Normals are estimated automatically if not already present
target_point_cloud	datatypes.PointCloud	required	The point cloud to align to. Normals are estimated automatically if not already present
initial_transformation_matrix	datatypes.Mat4x4 | np.ndarray | list[list[float]]	np.eye(4)	A 4x4 transform applied to source_point_cloud before ICP starts — should already bring it within max_correspondence_distance of target_point_cloud
max_iterations	datatypes.Int | int	50	Maximum number of ICP iterations to run
max_correspondence_distance	datatypes.Float | float | int	0.05	Maximum distance, in meters, at which two points are considered a match
normal_max_neighbors	datatypes.Int | int	30	Maximum number of neighbors used for normal estimation (when normals aren't already present)
normal_search_radius	datatypes.Float | float | int	0.05	Search radius, in meters, used for normal estimation
use_robust_kernel	datatypes.Bool | bool	False	Whether to down-weight outlier correspondences using the robust loss specified by loss_type, instead of the standard L2 loss
loss_type	datatypes.String | str	"L2"	The loss function used to weight correspondence errors when use_robust_kernel is True (ignored otherwise)
noise_standard_deviation	datatypes.Float | float | int	0.0	Expected standard deviation of point noise, in meters, used to weight correspondences by confidence

Returns

Type	Description
datatypes.Mat4x4	The 4x4 transform found by ICP — this is the transform itself, not an already-transformed point cloud. Pass it to apply_transform_to_point_cloud (as transformation_matrix) to actually move source_point_cloud's points. Use .data for the raw (4, 4) array.

Raises

Exception	Condition
TypeError	A parameter's value does not match its expected type (see the Parameter Configuration table above)
ValueError	initial_transformation_matrix is not shape (4, 4) (or, for a list input, doesn't contain only numeric elements)
ConfigurationError	The TELEKINESIS_API_KEY environment variable is not set
SerializationError	The request input failed to serialize, or the response failed to deserialize
RequestTimeoutError	The request to the Vitreous service timed out
TransportError	A network failure occurred before a response was received
ClientError	The Vitreous service rejected the request due to invalid input, invalid data, or another unexpected 4xx response
AuthenticationError	The API key was rejected as invalid or expired
AuthenticationServiceError	The authentication service was unavailable
ServerError	The Vitreous service returned a 5xx or otherwise unexpected error response

How to Tune the Parameters

The register_point_clouds_using_point_to_plane_icp Skill exposes seven tunable parameters that control the ICP iteration budget, matching distance, normal estimation, and outlier robustness.

initial_transformation_matrix

• Controls: The starting transform applied to source_point_cloud before ICP begins.
• Default: np.eye(4) (identity)
• Must already bring source_point_cloud within max_correspondence_distance of target_point_cloud — this Skill does not search for a rough alignment itself.

max_iterations

• Controls: The maximum number of ICP iterations to run.
• Units: Iterations (integer)
• Default: 50
• Increase → more refinement, but slower
• Decrease → faster, but may stop before fully converging
• Typical range: 10-200 — use 10-30 for fast, 30-50 for balanced, 50-200 for high precision

max_correspondence_distance

• Controls: The maximum distance, in meters, at which two points are considered a match.
• Units: Meters
• Default: 0.05
• Increase → allows matching more distant points, risking incorrect matches
• Decrease → requires closer matches
• Set to roughly 2-5x the point spacing
• Typical range: 0.01-0.1 meters

normal_max_neighbors

• Controls: The maximum number of neighbors used for normal estimation, when normals aren't already present on the point clouds.
• Units: Points (integer count)
• Default: 30
• Increase → smoother, more stable normals, but slower
• Decrease → faster
• Typical range: 10-50

normal_search_radius

• Controls: The search radius, in meters, used for normal estimation.
• Units: Meters
• Default: 0.05
• Increase → considers more neighbors
• Decrease → faster
• Set to roughly 2-5x the point spacing
• Typical range: 0.01-0.1 meters

use_robust_kernel

• Controls: Whether correspondence errors are down-weighted using the robust loss in loss_type, instead of the standard L2 loss.
• Default: False
• Set to True when the point clouds contain outliers or noisy points that could otherwise skew the alignment.

loss_type

• Controls: The loss function used to weight correspondence errors when use_robust_kernel is True (ignored otherwise).
• Default: "L2"
• Options observed in practice include "L2" (standard, no robustness), "L1", "Huber", and "tukey_loss" (each increasingly tolerant of outliers). The exact set of supported names is not enumerated in the SDK's own docstring — this is a documented gap in the source itself, not an omission here. If you need a specific robust loss and aren't sure it's supported, verify against your server's documentation before relying on it in production.

noise_standard_deviation

• Controls: The expected standard deviation of point noise, in meters, used to weight correspondences by confidence.
• Units: Meters
• Default: 0.0
• Increase → gives less weight to noisier correspondences
• Typical range: 0.0-0.01 — use 0.0 to disable, 0.001-0.005 for typical sensor noise

TIP

Point-to-plane ICP needs reliable normals to outperform point-to-point ICP — if normal_max_neighbors/normal_search_radius are too small for your point spacing, the estimated normals will be noisy and convergence quality will suffer. When your data has outliers, turn on use_robust_kernel before reaching for a smaller max_correspondence_distance.

Where to Use the Skill

Common pipelines include:

• Fine alignment refinement – tightening a coarse registration (from register_point_clouds_using_fast_global_registration or a sampler-based ICP Skill) into a precise final transform
• Precise 6D pose estimation – computing an accurate object pose for robotic grasping once a rough alignment is available
• Surface-based registration – aligning scans of smooth or planar parts where normals are reliable and point-to-point distance alone converges poorly
• Quality inspection – measuring fine deviations between a scanned part and its reference model after alignment

Alternative Skills

Skill	vs. Register Point Clouds Using Point to Plane ICP
register_point_clouds_using_point_to_point_icp	Matches by raw point-to-point distance instead of distance-to-tangent-plane. Use it when normals are unavailable or unreliable; use point-to-plane ICP when normals are available, since it typically converges faster and more accurately.
register_point_clouds_using_fast_global_registration	Feature-based registration that doesn't require rough pre-alignment. Run it first if the clouds aren't already roughly positioned within max_correspondence_distance.
register_point_clouds_using_cuboid_translation_sampler_icp	Grid-searches translations with repeated ICP runs to establish a rough alignment. An alternative coarse pre-alignment step when only the translation is uncertain.
register_point_clouds_using_rotation_sampler_icp	Grid-searches rotations with repeated ICP runs. An alternative coarse pre-alignment step when only the rotation is uncertain.

The SDK's register_point_clouds_using_centroid_translation is the fastest coarse pre-alignment option among the ones this Skill's docstring recommends, useful when only a translation offset (not a rotation) separates the clouds.

When Not to Use the Skill

Do not use Register Point Clouds Using Point to Plane ICP when:

• The point clouds aren't already roughly aligned within max_correspondence_distance – run a coarse method first, such as the SDK's register_point_clouds_using_centroid_translation, register_point_clouds_using_cuboid_translation_sampler_icp, register_point_clouds_using_rotation_sampler_icp, or register_point_clouds_using_fast_global_registration
• Normals are unavailable or unreliable (e.g. very sparse or noisy point clouds) – use register_point_clouds_using_point_to_point_icp instead
• The surfaces are not smooth or well-defined – the tangent-plane metric works best on smooth surfaces; use point-to-point ICP for irregular or highly detailed geometry
• You need the fastest possible registration – point-to-plane ICP does extra work (normal estimation and tangent-plane distances) compared to point-to-point ICP

TIP

If you're unsure whether your point clouds already have normals, you don't need to compute them yourself — this Skill estimates normals automatically using normal_max_neighbors/normal_search_radius whenever they're missing.