COCOObjectDetectionAnnotations

Represents a batch of ground-truth annotations in the COCO annotations format.

Parameters

Field	Type	Description
ids	list[int] | np.ndarray	(N,) annotation ids.
image_ids	list[int] | np.ndarray	(N,) source image ids.
category_ids	list[int] | np.ndarray	(N,) category ids.
bboxes	list[list[float]] | np.ndarray	(N, 4) boxes in [x, y, w, h] format. Required (unlike COCOObjectDetectionResults, there's no bbox-vs-segmentation "at least one" rule).
image_heights	list[int] | np.ndarray	(N,) source image heights, each > 0.
image_widths	list[int] | np.ndarray	(N,) source image widths, each > 0.
segmentations	list[Any] | np.ndarray | None	Optional length-N list, each entry a polygon, an RLE dict, or None. Normalized to encoded COCO RLE. See SegmentationFormat.

Raises

Exception	Condition
ValueError	bboxes isn't shape (N, 4) (N = bboxes.shape[0]).
ValueError	Any of ids/image_ids/category_ids/image_heights/image_widths/segmentations doesn't have length N.
ValueError	image_heights/image_widths contains a value <= 0.
TypeError	An entry in segmentations is neither None, a valid polygon list, nor an RLE dict with size/counts keys.
ValueError	An entry in segmentations is an already-encoded RLE dict whose size doesn't match that row's (image_height, image_width).

Attributes

Attribute	Type	Description
ids	np.ndarray	Defensive copy, shape (N,) int32, annotation ids.
image_ids	np.ndarray	Defensive copy, shape (N,) int32, source image ids.
category_ids	np.ndarray	Defensive copy, shape (N,) int32, category ids.
bboxes	np.ndarray	Defensive copy, shape (N, 4) float32, [x, y, w, h]. Always present.
image_heights	np.ndarray	Defensive copy, shape (N,) int32, source image heights.
image_widths	np.ndarray	Defensive copy, shape (N,) int32, source image widths.
segmentations	list[dict[str, Any] | None] | None	Defensive copy: a canonical encoded COCO RLE dict or None per entry, or None if no segmentations were provided.
masks	np.ndarray	Decodes every segmentation into one (N, H, W) bool array (one plane per annotation; entries without a segmentation are all-False). Assumes every annotation shares the same (H, W) -- taken from row 0's image_height/image_width. Returns shape (0, 0, 0) when the batch is empty.

Methods

Method	Description
COCOObjectDetectionAnnotations.coerce(value)	Returns value unchanged if it's already a COCOObjectDetectionAnnotations; if it's a dict, constructs one via COCOObjectDetectionAnnotations(**value). Raises TypeError for any other input.
COCOObjectDetectionAnnotations.from_binary_masks(ids, image_ids, category_ids, bboxes, masks)	Alternate constructor: builds a batch from a non-empty list of N binary (H, W) masks (heights/widths may differ per mask). Each mask's own shape supplies image_heights/image_widths, and each mask is encoded to COCO RLE to populate segmentations. Not available on COCOObjectDetectionResults. Raises ValueError if masks is empty, or if any array's length doesn't match N = len(masks).
anns.get_segmentation(idx, target_type)	Returns entry idx's segmentation converted to target_type (a SegmentationFormat member or its string value), or None if there are no segmentations or entry idx has none.
COCOObjectDetectionAnnotations.convert_segmentation(segmentation, source_type, target_type, *, height=None, width=None, min_points=3, epsilon=0.0)	Converts a segmentation between "rle", "polygon", and "mask" representations. height/width are required only when source_type is "polygon". min_points/epsilon only affect conversion to "polygon". A stateless class method shared with COCOObjectDetectionAnnotation, COCOObjectDetectionResult, and COCOObjectDetectionResults. Raises ImportError if converting to "polygon" and OpenCV isn't installed.

Operators

Operation	Behavior
len(anns)	Number of annotations N in the batch.
anns[i]	An int returns a single COCOObjectDetectionAnnotation. A slice or boolean np.ndarray mask returns a new COCOObjectDetectionAnnotations sub-batch. Raises IndexError for an out-of-range int, ValueError for a boolean mask of the wrong length, TypeError for any other index type.
a == b	True only if b is also a COCOObjectDetectionAnnotations with equal ids, image_ids, category_ids, bboxes, image_heights, image_widths, and segmentations; NotImplemented if b isn't a COCOObjectDetectionAnnotations.
hash(anns)	Not supported (__hash__ = None), despite the object being immutable after construction.

Serialization

to_pyarrow/from_pyarrow round-trip through a length-1 pa.StructArray whose fields are each a length-1 pa.ListArray wrapping the full length-N batch:

StructArray length 1
├── ids:                  list<int32>                        (inner length N)
├── image_ids:            list<int32>                        (inner length N)
├── category_ids:         list<int32>                        (inner length N)
├── bboxes:               list<fixed_size_list<float32>[4]>  (inner length N)
├── image_heights:        list<int32>                        (inner length N)
├── image_widths:         list<int32>                        (inner length N)
├── segmentation_heights: list<int32>                        (inner length N, 0 per row without a segmentation)
├── segmentation_widths:  list<int32>                        (inner length N, 0 per row without a segmentation)
├── segmentation_valid:   list<bool>                         (inner length N; True iff that row has a mask)
└── segmentation_counts:  list<string>                       (inner length N, null per row without a segmentation)

Unlike COCOObjectDetectionResults, which relies solely on a null segmentation_counts entry as the per-row validity marker, this class also carries an explicit segmentation_valid boolean column.

Visualization

datatypes.visualize(anns, entity_path=...) logs one batched rr.Boxes2D at {entity_path}/boxes (boxes are always present) with class_ids=category_ids and labels=[str(id) for id in ids] (annotation ids, not category labels or scores -- there's no score field). For every non-None entry in segmentations, additionally decodes the RLE and logs an rr.SegmentationImage at {entity_path}/masks/mask{i}, each with its own two-class rr.AnnotationContext (transparent background, foreground labeled "cat={category_id}").

Example

"""Demonstrates the Telekinesis COCOObjectDetectionAnnotations datatype."""

import time

import numpy as np
from loguru import logger
import rerun as rr

from telekinesis import datatypes

def object_detection_annotations_example():
    """Demonstrate creation, access, indexing, segmentation format conversion, construction from masks, and serialization."""

    # ======================= Create ============================================
    H, W = 720, 1280
    annotations = datatypes.COCOObjectDetectionAnnotations(
        ids=np.array([0, 1], dtype=np.int32),
        image_ids=np.array([7, 7], dtype=np.int32),
        category_ids=np.array([1, 2], dtype=np.int32),
        bboxes=np.array([[0, 0, 10, 10], [20, 20, 5, 5]], dtype=np.float32),
        image_heights=np.array([H, H], dtype=np.int32),
        image_widths=np.array([W, W], dtype=np.int32),
        segmentations=[
            [[0, 2, 10, 0, 10, 10, 0, 10]],
            [[22, 20, 25, 20, 25, 25, 20, 25]],
        ],
    )
    logger.info(f"Original COCOObjectDetectionAnnotations: {annotations}")

    # ======================= Inspect ===========================================
    logger.info(f"Number of annotations in batch: {len(annotations)}")
    logger.info(
        f"ids={annotations.ids}, "
        f"image_ids={annotations.image_ids}, "
        f"category_ids={annotations.category_ids}, "
        f"image_heights={annotations.image_heights}, "
        f"image_widths={annotations.image_widths}"
    )
    logger.info(f"bboxes={annotations.bboxes}")
    logger.info(f"segmentations={annotations.segmentations}")

    # ======================= Visualize =========================================
    rr.init("object_detection_example", spawn=True)
    datatypes.visualize(annotations, entity_path="/COCOObjectDetectionAnnotations")

    # ======================= Index =============================================
    index = 0
    single = annotations[index]
    logger.info(f"Single ObjectDetectionAnnotation at index {index}: {single}")
    logger.info(
        f"id={single.id}, "
        f"image_id={single.image_id}, "
        f"category_id={single.category_id}, "
        f"bbox={single.bbox}"
    )
    logger.info(f"segmentation={single.segmentation}")
    datatypes.visualize(single, entity_path="/SingleObjectDetectionAnnotation")

    # ======================= Convert ===========================================
    as_rle = annotations.segmentations[0]
    as_polygon = annotations.get_segmentation(0, "polygon")
    as_mask = annotations.get_segmentation(0, "mask")
    logger.info(f"Segmentation 0 as RLE: {as_rle}")
    logger.info(f"Segmentation 0 as polygon: {as_polygon}")
    logger.info(f"Segmentation 0 as mask: shape={as_mask.shape}, dtype={as_mask.dtype}")

    mask_back_to_rle = datatypes.COCOObjectDetectionAnnotations.convert_segmentation(
        as_mask, "mask", "rle"
    )
    logger.info(f"Mask converted back to RLE: {mask_back_to_rle}")

    # ======================= From Masks ========================================
    masks = [
        annotations.get_segmentation(i, datatypes.SegmentationFormat.MASK)
        for i in range(len(annotations))
    ]
    from_masks = datatypes.COCOObjectDetectionAnnotations.from_binary_masks(
        ids=annotations.ids,
        image_ids=annotations.image_ids,
        category_ids=annotations.category_ids,
        bboxes=annotations.bboxes,
        masks=masks,
    )
    logger.info(f"Built from binary masks: {from_masks}")

    # ======================= Serialize / Deserialize ===========================
    start = time.perf_counter()
    serialized = datatypes.serialize(annotations)
    serialization_ms = (time.perf_counter() - start) * 1000

    start = time.perf_counter()
    deserialized = datatypes.deserialize(serialized)["param_0"]
    deserialization_ms = (time.perf_counter() - start) * 1000

    logger.info(f"Deserialized COCOObjectDetectionAnnotations: {deserialized}")
    logger.info(f"Round-trip successful: {annotations == deserialized}")
    logger.info(f"Serialization time: {serialization_ms:.3f} ms")
    logger.info(f"Deserialization time: {deserialization_ms:.3f} ms")


if __name__ == "__main__":
    object_detection_annotations_example()