Project Camera Point To Pixel

SUMMARY

Project Camera Point To Pixel projects a 3D point in camera coordinates to pixel coordinates.

It is the forward pinhole-camera projection: given a point [x, y, z] in the camera frame (camera at the origin, looking down +z), camera_intrinsics, and distortion_coefficients, it computes the [u, v] pixel it maps to. Use project_pixel_to_camera_point for the inverse — pixel + depth back to a 3D camera-space point — or project_world_point_to_pixel if the point is in world coordinates rather than camera coordinates.

Use this Skill when you want to convert a 3D point already in camera coordinates into a pixel location.

The Skill

from telekinesis import pupil
import numpy as np

# [[fx, 0, cx], [0, fy, cy], [0, 0, 1]]
camera_intrinsics = np.array(
    [[500.0, 0, 320.0], [0, 500.0, 240.0], [0, 0, 1.0]], dtype=np.float64
)
# [k1, k2, p1, p2, k3]
distortion_coefficients = np.array([0.0, 0.0, 0.0, 0.0, 0.0], dtype=np.float64)
# [x, y, z] in camera coordinates
point = np.array([0.0, 0.0, 1.0], dtype=np.float64)

pixel = pupil.project_camera_point_to_pixel(
    camera_intrinsics=camera_intrinsics,
    distortion_coefficients=distortion_coefficients,
    point=point,
)

API Reference

Full parameter and return type documentation for project_camera_point_to_pixel.

View Reference →

Example

Projecting the camera-frame point [0.0, 0.0, 1.0] (one unit directly in front of the camera, on the optical axis) through intrinsics [[500, 0, 320], [0, 500, 240], [0, 0, 1]] with zero distortion yields the pixel [320.0, 240.0] — the principal point, as expected for a point on the optical axis. There is no before/after image for this Skill: to verify a projection visually, draw a marker at the returned (u, v) on the corresponding image, or compare it against a known pixel correspondence from calibration.

The Code

"""Demonstrates projecting a 3D camera point to pixel coordinates."""

import numpy as np
from loguru import logger
import rerun as rr

from telekinesis import pupil, datatypes


def project_camera_point_to_pixel_example():
    """Projects a 3D camera point to pixel coordinates."""
    # ===================== Create Parameters ==========================================
    # Point: [x, y, z] in camera coordinates
    point = datatypes.Point3D(np.array([0.0, 0.0, 1.0], dtype=np.float64))
    
    # Camera_intrinsics: [[fx, 0, cx], [0, fy, cy], [0, 0, 1]]
    camera_intrinsics = np.array(
        [[500.0, 0, 320.0],
         [0, 500.0, 240.0],
         [0, 0, 1.0]],
        dtype=np.float64,
    )
    # Distortion coefficients: [k1, k2, p1, p2, k3]
    distortion_coefficients = np.array(
        [0.0, 0.0, 0.0, 0.0, 0.0], dtype=np.float64
    )

    # ===================== Run Skill ==========================================
    pixel = pupil.project_camera_point_to_pixel(
        point=point,
        camera_intrinsics=camera_intrinsics,
        distortion_coefficients=distortion_coefficients,
    )

    # ===================== Log ================================================
    logger.success(f"Projected camera point to pixel using {point}")
    logger.success(f"Result: {pixel}")

    # ===================== Visualization  (Optional) ======================
    rr.init("project_camera_point_to_pixel_example", spawn=True)
    datatypes.visualize(point, entity_path="0-Point in 3D Space")
    datatypes.visualize(pixel, entity_path="1-Pixel")

if __name__ == "__main__":
    project_camera_point_to_pixel_example()

Runnable examples are available in the Telekinesis examples repository.

Follow the README in that repository to set up the environment, run this specific example with:

cd telekinesis-examples
python examples/image_processing/project_camera_point_to_pixel.py

Parameter Configuration

Key	Type	Default	Description
point	datatypes.Point3D | np.ndarray | list	required	The 3D point [x, y, z] in camera coordinates (camera at the origin, looking down +z), shape (3,)
camera_intrinsics	datatypes.Mat3x3 | np.ndarray | list	required	3x3 camera intrinsic matrix [[fx, 0, cx], [0, fy, cy], [0, 0, 1]], shape (3, 3)
distortion_coefficients	datatypes.Array | np.ndarray | list	required	Lens distortion coefficients [k1, k2, p1, p2, k3]. Pass all zeros for an undistorted/ideal pinhole model

Returns

Type	Description
datatypes.Point2D	The projected pixel coordinates [x, y]. Access the raw (2,) array via .data

Raises

Exception	Condition
TypeError	A parameter's value does not match its expected type (see the Parameter Configuration table above)
ValueError	distortion_coefficients is not shape (5,), or contains NaN/inf
ConfigurationError	The TELEKINESIS_API_KEY environment variable is not set
SerializationError	The request input failed to serialize, or the response failed to deserialize
RequestTimeoutError	The request to the Pupil service timed out
TransportError	A network failure occurred before a response was received
ClientError	The Pupil service rejected the request due to invalid input, invalid data, or another unexpected 4xx response
AuthenticationError	The API key was rejected as invalid or expired
AuthenticationServiceError	The authentication service was unavailable
ServerError	The Pupil service returned a 5xx or otherwise unexpected error response

How to Tune the Parameters

These are camera-geometry inputs rather than tunable knobs in the usual sense — there is no "better" or "worse" value, only the value that matches your camera and the point you want projected.

point

• Controls: The 3D location, in camera coordinates, to project.
• Practical guidance: Must have z > 0 to lie in front of the camera; a point with z <= 0 projects to a meaningless pixel.

camera_intrinsics

• Controls: The focal lengths (fx, fy) and principal point (cx, cy) of the camera.
• Practical guidance: Obtain from a camera calibration procedure. Must match the resolution/units the point and downstream pixel consumers expect.

distortion_coefficients

• Controls: Lens distortion correction applied during projection.
• Default: [0.0, 0.0, 0.0, 0.0, 0.0] (no distortion, ideal pinhole model)
• Practical guidance: Use the 5-element [k1, k2, p1, p2, k3] vector from the same calibration that produced camera_intrinsics. Must be exactly shape (5,) with no NaN/inf values.

TIP

Best practice: Keep camera_intrinsics and distortion_coefficients paired from the same calibration run — mixing intrinsics from one calibration with distortion coefficients from another produces silently wrong pixel locations.

Where to Use the Skill

Common pipelines include:

• 3D overlay – Draw a 3D point, keypoint, or object center on the corresponding camera image
• Pose verification – Project a 3D model or estimated pose back to the image plane to visually check correctness
• Rendering/AR – Convert 3D scene content into 2D for display or augmentation
• Annotation generation – Derive 2D bounding boxes or keypoint pixels from known 3D geometry

Alternative Skills

Skill	vs. Project Camera Point To Pixel
project_pixel_to_camera_point	The inverse operation: pixel + depth back to a 3D camera-space point.
project_world_point_to_pixel	Same forward projection, but the input point is in world coordinates and a world_T_camera transform is required.
project_pixel_to_world_point	Inverse operation into world coordinates instead of camera coordinates.

When Not to Use the Skill

Do not use Project Camera Point To Pixel when:

• The point is in world coordinates (use project_world_point_to_pixel, which additionally applies world_T_camera)
• You have a pixel and depth and need a 3D point (use project_pixel_to_camera_point, the inverse operation)
• The point may have z <= 0 (points behind or at the camera do not project meaningfully; check z > 0 first)
• You don't have a calibrated camera_intrinsics/distortion_coefficients pair (calibrate the camera first; guessed values produce silently wrong pixels)