COCOObjectDetectionAnnotations

SUMMARY

A batch of COCO-style object detection or instance segmentation annotations.

from telekinesis import datatypes
annotations = datatypes.COCOObjectDetectionAnnotations(
    ids=[0, 1],
    image_ids=[7, 7],
    category_ids=[1, 2],
    bboxes=[[0, 0, 10, 10], [20, 20, 5, 5]],
    areas=[100.0, 25.0],
    iscrowds=[False, False],
)

Segmentation representation follows iscrowds

Entry i with iscrowds[i]=False stores that row's segmentations[i] as a native polygon; iscrowds[i]=True stores it as an encoded COCO RLE dict. Call as_masks() to rasterize every row to a binary mask.

API Reference

Complete API documentation for COCOObjectDetectionAnnotations, including parameters, attributes, and methods.

View Reference →

Parameters

Parameter	Type	Default	Description
ids	list[int] | np.ndarray	Required	Annotation identifiers with shape (N,).
image_ids	list[int] | np.ndarray	Required	Source image identifiers with shape (N,).
category_ids	list[int] | np.ndarray	Required	Annotated category identifiers with shape (N,).
bboxes	list[list[float]] | np.ndarray	Required	Bounding boxes with shape (N, 4), with one [x, y, width, height] row per annotation. Width and height must be non-negative.
areas	list[float] | np.ndarray	Required	Annotation areas with shape (N,). Each value must be greater than or equal to 0.
iscrowds	list[bool] | np.ndarray	Required	Crowd flags with shape (N,). Each flag determines the expected segmentation representation for the corresponding annotation.
segmentations	list[COCOPolygonSegmentationLike| COCORLESegmentationLike | None] | None	None	Length-N sequence of polygon or COCO RLE segmentations, or None entries. Each representation must agree with the corresponding iscrowds value.

Raises

Exception	Condition
TypeError	A segmentations entry is present but structurally doesn't match its corresponding iscrowds entry.
ValueError	bboxes isn't shape (N, 4) (N = bboxes.shape[0]), contains a non-finite value, or has negative width/height for a row.
ValueError	Any of ids/image_ids/category_ids/areas/iscrowds/segmentations doesn't have length N.
ValueError	areas contains a negative or non-finite value.
ValueError	A segmentations entry fails RLE/polygon value validation.

Attributes

Attribute	Type	Description
ids	np.ndarray	Defensive copy, shape (N,) int32, annotation ids.
image_ids	np.ndarray	Defensive copy, shape (N,) int32, source image ids.
category_ids	np.ndarray	Defensive copy, shape (N,) int32, category ids.
bboxes	np.ndarray	Defensive copy, shape (N, 4) float32, [x, y, w, h]. Always present.
areas	np.ndarray	Defensive copy, shape (N,) float32, annotation areas.
iscrowds	np.ndarray	Defensive copy, shape (N,) bool, crowd flags; selects each entry's segmentation representation.
segmentations	list[COCOPolygonSegmentation| COCORLESegmentation | None] | None	Defensive copy: a polygon (where iscrowds[i] is False), a canonical RLE dict (where iscrowds[i] is True), or None per entry, or None if no segmentations were provided.

Methods

Method	Type	Description
COCOObjectDetectionAnnotations.coerce(value)	COCOObjectDetectionAnnotations	Converts a dict of constructor arguments into a COCOObjectDetectionAnnotations. If value is already a COCOObjectDetectionAnnotations, it is returned unchanged.
COCOObjectDetectionAnnotations.from_masks(*, ids, image_ids, category_ids, bboxes, areas, masks)	COCOObjectDetectionAnnotations	Builds a batch from a length-N list of binary (H, W) masks, encoding each one as an RLE segmentation. Every entry in masks must be an actual mask, not None, and the resulting batch always has every iscrowds entry True.
as_masks(image_heights=..., image_widths=...)	COCOObjectDetectionAnnotationsAsMasks	Returns this batch with every segmentation rasterized to a binary mask at its corresponding per-row size. image_heights/image_widths must each have length N; for an iscrowd row the given size must match the size already embedded in its RLE, and for a polygon row it must be a positive integer. See COCOObjectDetectionAnnotationsAsMasks for the returned fields.
COCOObjectDetectionAnnotations.mask_to_rle(mask)	COCORLESegmentation	Encodes a binary (H, W) mask as RLE, returning a canonical COCORLESegmentation.
COCOObjectDetectionAnnotations.polygon_to_rle(polygon, *, height, width)	COCORLESegmentation	Rasterizes one or more polygons (a COCOPolygonSegmentationLike) and encodes the result as a canonical COCORLESegmentation.
COCOObjectDetectionAnnotations.rle_to_mask(rle)	np.ndarray	Decodes a COCORLESegmentationLike into a binary (H, W) mask (uint8).
COCOObjectDetectionAnnotations.rle_to_polygon(rle, *, min_points=3, epsilon=0.0)	COCOPolygonSegmentation	Approximates a COCORLESegmentationLike as a COCOPolygonSegmentation by tracing the mask's boundary with marching squares.

Operators

Operation	Behavior
len(anns)	Number of annotations N in the batch.
anns[i]	An int returns a single COCOObjectDetectionAnnotation. A slice or boolean np.ndarray mask returns a new COCOObjectDetectionAnnotations sub-batch. Raises IndexError for an out-of-range int, ValueError for a boolean mask of the wrong length, TypeError for any other index type.
a == b	True only if b is also a COCOObjectDetectionAnnotations with equal ids, image_ids, category_ids, bboxes, areas, iscrowds, and segmentations; NotImplemented if b isn't a COCOObjectDetectionAnnotations.

Representations

Representation	Method	Result
Masks	as_masks(image_heights=..., image_widths=...)	COCOObjectDetectionAnnotationsAsMasks

Visualization

import rerun as rr

# Your code block
# ....

rr.init("coco_object_detection_annotations_example", spawn=True)
datatypes.visualize(annotations, entity_path="/annotations", label="COCOObjectDetectionAnnotations")

Example

"""Demonstrates the Telekinesis COCOObjectDetectionAnnotations datatype."""

import time

import numpy as np
import rerun as rr
from loguru import logger

from telekinesis import datatypes

def coco_object_detection_annotations_example():
    """Demonstrate creation, inspection, operations, visualization, and serialization."""

    # ======================= Create ============================================
    # Segmentation semantics: iscrowd=False -> polygon (stored natively, not
    # converted), iscrowd=True -> encoded COCO RLE. The two representations are
    # never silently converted between each other.
    image_height, image_width = 720, 1280
    annotations = datatypes.COCOObjectDetectionAnnotations(
        ids=np.array([0, 1], dtype=np.int32),
        image_ids=np.array([7, 7], dtype=np.int32),
        category_ids=np.array([1, 2], dtype=np.int32),
        bboxes=np.array([[0, 0, 10, 10], [20, 20, 5, 5]], dtype=np.float32),
        areas=np.array([100.0, 25.0], dtype=np.float32),
        iscrowds=np.array([False, False]),
        segmentations=[
            [[0, 2, 10, 0, 10, 10, 0, 10]],
            [[22, 20, 25, 20, 25, 25, 20, 25]],
        ],
    )
    logger.info(f"Created COCOObjectDetectionAnnotations: {annotations}")

    # ======================= Inspect ===========================================
    logger.info(f"Number of annotations in batch: {len(annotations)}")
    logger.info(f"ids={annotations.ids}")
    logger.info(f"image_ids={annotations.image_ids}")
    logger.info(f"category_ids={annotations.category_ids}")
    logger.info(f"bboxes={annotations.bboxes}")
    logger.info(f"areas={annotations.areas}")
    logger.info(f"iscrowds={annotations.iscrowds}")
    logger.info(f"segmentations={annotations.segmentations}")

    # ======================= Operations =========================================
    index = 0
    single_annotation = annotations[index]
    logger.info(f"COCOObjectDetectionAnnotation at index {index}: {single_annotation}")

    sliced_annotations = annotations[0:1]
    logger.info(f"Sliced COCOObjectDetectionAnnotations: {sliced_annotations}")

    keep_mask = np.array([True, False])
    masked_annotations = annotations[keep_mask]
    logger.info(f"Masked COCOObjectDetectionAnnotations: {masked_annotations}")

    # Neither the single annotation nor the batch stores an image size, so
    # `as_mask`/`as_masks` take the target size explicitly. For an `iscrowd=True`
    # (RLE) entry, the passed size must match the RLE's own embedded size.
    single_as_mask = single_annotation.as_mask(image_height=image_height, image_width=image_width)
    logger.info(
        f"Segmentation 0 as mask: shape={single_as_mask['segmentation'].shape}, "
        f"dtype={single_as_mask['segmentation'].dtype}"
    )

    annotations_as_masks = annotations.as_masks(
        image_heights=[image_height, image_height], image_widths=[image_width, image_width]
    )
    logger.info(f"All masks: shapes={[m.shape for m in annotations_as_masks['segmentations']]}")

    # `from_masks` always sets `iscrowd=True` for every row. `areas` must
    # still be passed explicitly, since it can't be derived from the masks.
    # Every entry in `masks` must be a real mask. For rows without a mask,
    # build them individually with `COCOObjectDetectionAnnotation.from_mask`
    # or the plain constructor instead.
    crowd_mask_0 = np.zeros((image_height, image_width), dtype=np.uint8)
    crowd_mask_0[100:200, 150:400] = 1
    crowd_mask_1 = np.zeros((image_height, image_width), dtype=np.uint8)
    crowd_mask_1[300:350, 200:250] = 1
    crowd_annotations = datatypes.COCOObjectDetectionAnnotations.from_masks(
        ids=[2, 3],
        image_ids=[7, 7],
        category_ids=[3, 3],
        bboxes=[[150, 100, 250, 100], [200, 300, 50, 50]],
        areas=[float(crowd_mask_0.sum()), float(crowd_mask_1.sum())],
        masks=[crowd_mask_0, crowd_mask_1],
    )
    logger.info(f"Crowd annotations batch built from masks: {crowd_annotations}")

    # Mixin helpers shared across all COCO segmentation datatypes.
    mask_from_rle = datatypes.COCOObjectDetectionAnnotations.rle_to_mask(
        crowd_annotations.segmentations[0]
    )
    logger.info(f"Mask decoded via rle_to_mask: shape={mask_from_rle.shape}, dtype={mask_from_rle.dtype}")

    polygon_from_rle = datatypes.COCOObjectDetectionAnnotations.rle_to_polygon(
        crowd_annotations.segmentations[0]
    )
    logger.info(f"Polygon decoded via rle_to_polygon: {polygon_from_rle}")

    rle_from_polygon = datatypes.COCOObjectDetectionAnnotations.polygon_to_rle(
        single_annotation.segmentation, height=image_height, width=image_width
    )
    logger.info(f"RLE encoded via polygon_to_rle: {rle_from_polygon}")

    # ======================= Visualize =========================================
    rr.init("coco_object_detection_annotations_example", spawn=True)
    datatypes.visualize(annotations, entity_path="/coco_object_detection_annotations")
    datatypes.visualize(single_annotation, entity_path="/coco_object_detection_annotations/single")
    datatypes.visualize(crowd_annotations, entity_path="/coco_object_detection_annotations/crowd")

    # ======================= Serialize / Deserialize ===========================
    start = time.perf_counter()
    serialized = datatypes.serialize(annotations)
    serialization_ms = (time.perf_counter() - start) * 1000

    start = time.perf_counter()
    deserialized = datatypes.deserialize(serialized)["param_0"]
    deserialization_ms = (time.perf_counter() - start) * 1000

    logger.info(f"Deserialized COCOObjectDetectionAnnotations: {deserialized}")
    logger.info(f"Round-trip successful: {annotations == deserialized}")
    logger.info(f"Serialization time: {serialization_ms:.3f} ms")
    logger.info(f"Deserialization time: {deserialization_ms:.3f} ms")


if __name__ == "__main__":
    coco_object_detection_annotations_example()