COCOObjectDetectionResults

Represents a batch of predictions in the COCO "results" format: per-image, per-category detections with confidence scores.

Parameters

Field	Type	Description
image_ids	np.ndarray | list[int]	(N,) source image ids.
category_ids	np.ndarray | list[int]	(N,) category ids.
image_heights	np.ndarray | list[int]	(N,) source image heights, each > 0.
image_widths	np.ndarray | list[int]	(N,) source image widths, each > 0.
scores	np.ndarray | list[float]	(N,) detection scores, each >= 0.
bboxes	np.ndarray | list[list[float]] | None	Optional (N, 4) boxes in [x, y, w, h] format. At least one of bboxes/segmentations must be given.
segmentations	list[dict[str, Any] | list | None] | None	Optional length-N list, each entry a polygon, an RLE dict, or None. Normalized to encoded COCO RLE. See SegmentationFormat.

Raises

Exception	Condition
ValueError	Both bboxes and segmentations are None.
ValueError	bboxes isn't shape (N, 4), or any of image_ids/category_ids/image_heights/image_widths/scores/segmentations doesn't have length N (N = len(image_ids)).
ValueError	image_heights/image_widths contains a value <= 0.
ValueError	scores contains a negative value.
TypeError	An entry in segmentations is neither None, a valid polygon list, nor an RLE dict with size/counts keys.
ValueError	An entry in segmentations is an already-encoded RLE dict whose size doesn't match that row's (image_height, image_width).

Attributes

Attribute	Type	Description
image_ids	np.ndarray	Defensive copy, shape (N,) int32, source image ids.
category_ids	np.ndarray	Defensive copy, shape (N,) int32, category ids.
image_heights	np.ndarray	Defensive copy, shape (N,) int32, source image heights.
image_widths	np.ndarray	Defensive copy, shape (N,) int32, source image widths.
scores	np.ndarray | None	Defensive copy, shape (N,) float32, detection scores. None only for an empty (N=0) batch -- the constructor requires scores, but an empty array is normalized to None internally.
bboxes	np.ndarray | None	Defensive copy, shape (N, 4) float32, [x, y, w, h], or None if no boxes were provided.
segmentations	list[dict[str, Any] | None] | None	Defensive copy: a canonical encoded COCO RLE dict or None per entry, or None if no segmentations were provided.

Methods

Method	Description
COCOObjectDetectionResults.coerce(value)	Returns value unchanged if it's already a COCOObjectDetectionResults; if it's a dict, constructs one via COCOObjectDetectionResults(**value). Raises TypeError for any other input.
results.get_segmentation(idx, target_type)	Returns entry idx's segmentation converted to target_type (a SegmentationFormat member or its string value), or None if there are no segmentations or entry idx has none. Entries are always stored as canonical RLE, so only the target format needs to be specified.
COCOObjectDetectionResults.convert_segmentation(segmentation, source_type, target_type, *, height=None, width=None, min_points=3, epsilon=0.0)	Converts a segmentation between "rle", "polygon", and "mask" representations. height/width are required only when source_type is "polygon". min_points/epsilon only affect conversion to "polygon". A stateless class method shared with COCOObjectDetectionResult, COCOObjectDetectionAnnotation, and COCOObjectDetectionAnnotations. Raises ImportError if converting to "polygon" and OpenCV isn't installed.

Operators

Operation	Behavior
len(results)	Number of results N in the batch.
results[i]	An int returns a single COCOObjectDetectionResult. A slice or boolean np.ndarray mask returns a new COCOObjectDetectionResults sub-batch. Raises IndexError for an out-of-range int, ValueError for a boolean mask of the wrong length, TypeError for any other index type.
a == b	True only if b is also a COCOObjectDetectionResults with equal image_ids, category_ids, bboxes, image_heights, image_widths, scores, and segmentations; NotImplemented if b isn't a COCOObjectDetectionResults.
hash(results)	Not supported (__hash__ = None), despite the object being immutable after construction.

Serialization

to_pyarrow/from_pyarrow round-trip through a length-1 pa.StructArray whose fields are each a length-1 pa.ListArray wrapping the full length-N batch:

StructArray length 1
├── image_ids:            list<int32>                        (inner length N)
├── category_ids:         list<int32>                        (inner length N)
├── bboxes:               list<fixed_size_list<float32>[4]>  (inner length N; all-null when bboxes is None)
├── image_heights:        list<int32>                        (inner length N)
├── image_widths:         list<int32>                        (inner length N)
├── scores:               list<float32>                      (inner length N; empty when scores is None)
├── segmentation_heights: list<int32>                        (inner length N, 0 per row without a segmentation)
├── segmentation_widths:  list<int32>                        (inner length N, 0 per row without a segmentation)
└── segmentation_counts:  list<string>                       (inner length N, null per row without a segmentation)

As with the singular COCOObjectDetectionResult, each row's RLE size/counts are split into flat segmentation_heights/segmentation_widths/segmentation_counts columns rather than a nested struct, and a null entry in segmentation_counts is the sole per-row validity marker.

Visualization

datatypes.visualize(results, entity_path=...): when bboxes is present, logs one batched rr.Boxes2D at {entity_path}/boxes with class_ids=category_ids and labels "cat={category_id}", each suffixed with " ({score:.2f})" when scores is present. For every non-None entry in segmentations, additionally decodes the RLE and logs an rr.SegmentationImage at {entity_path}/masks/mask{i}, each with its own two-class rr.AnnotationContext (transparent background, foreground labeled the same "cat={category_id}" + score suffix).

Example

"""Demonstrates the Telekinesis COCOObjectDetectionResults datatype."""

import time

import rerun as rr
import numpy as np
from loguru import logger

from telekinesis import datatypes

def coco_object_detection_results_example():
    """Demonstrate creation, indexing, segmentation conversion, visualization, and serialization."""

    # ======================= Case 1: Bounding Boxes Only =======================
    H, W = 720, 1280
    bbox_results = datatypes.COCOObjectDetectionResults(
        image_ids=np.array([7, 7], dtype=np.int32),
        category_ids=np.array([1, 2], dtype=np.int32),
        image_heights=np.array([H, H], dtype=np.int32),
        image_widths=np.array([W, W], dtype=np.int32),
        scores=np.array([0.95, 0.82], dtype=np.float32),
        bboxes=np.array(
            [[0, 0, 10, 10], [20, 20, 5, 5]],
            dtype=np.float32,
        ),
    )
    logger.info(f"Original COCOObjectDetectionResults: {bbox_results}")

    # ======================= Inspect ===========================================
    image_ids = bbox_results.image_ids
    category_ids = bbox_results.category_ids
    image_heights = bbox_results.image_heights
    image_widths = bbox_results.image_widths
    scores = bbox_results.scores
    bboxes = bbox_results.bboxes
    segmentations = bbox_results.segmentations

    logger.info(f"Number of results in batch: {len(bboxes)}")
    logger.info(
        f"image_ids={image_ids}, "
        f"category_ids={category_ids}, "
        f"image_heights={image_heights}, "
        f"image_widths={image_widths}, "
        f"scores={scores}"
    )
    logger.info(f"Bboxes: {bboxes}")
    logger.info(f"Segmentations: {segmentations}")

    # ======================= Index =============================================
    index = 0
    detection_at_index = bbox_results[index]
    image_id = detection_at_index.image_id
    category_id = detection_at_index.category_id
    bbox = detection_at_index.bbox
    score = detection_at_index.score

    logger.info(
        f"ObjectDetectionResult at index {index}: "
        f"image_id={image_id}, "
        f"category_id={category_id}, "
        f"bbox={bbox}, "
        f"score={score}"
    )

    # ======================= Visualize =========================================
    rr.init("coco_object_detection_results_example", spawn=True)
    datatypes.visualize(bbox_results, entity_path="/COCOObjectDetectionResults")

    first_result = bbox_results[0]
    datatypes.visualize(first_result, entity_path="/COCOObjectDetectionResults/FirstResult")

    # ======================= Case 2: Bbox + Segmentation =======================
    bbox_segmentation_results = datatypes.COCOObjectDetectionResults(
        image_ids=np.array([7, 7], dtype=np.int32),
        category_ids=np.array([1, 2], dtype=np.int32),
        image_heights=np.array([H, H], dtype=np.int32),
        image_widths=np.array([W, W], dtype=np.int32),
        scores=np.array([0.95, 0.82], dtype=np.float32),
        bboxes=np.array([[0, 0, 10, 10], [20, 20, 5, 5]], dtype=np.float32),
        segmentations=[
            [[0, 2, 10, 0, 10, 10, 0, 10]],
            [[22, 20, 25, 20, 25, 25, 20, 25]],
        ],
    )
    datatypes.visualize(
        bbox_segmentation_results, entity_path="/COCOObjectDetectionResultsWithSegmentation"
    )

    # ======================= Case 3: Segmentation Only =========================
    segmentation_only_results = datatypes.COCOObjectDetectionResults(
        image_ids=np.array([7, 7], dtype=np.int32),
        category_ids=np.array([1, 2], dtype=np.int32),
        image_heights=np.array([H, H], dtype=np.int32),
        image_widths=np.array([W, W], dtype=np.int32),
        scores=np.array([0.95, 0.82], dtype=np.float32),
        segmentations=[
            [[0, 2, 10, 0, 10, 10, 0, 10]],
            [[22, 20, 25, 20, 25, 25, 20, 25]],
        ],
    )
    datatypes.visualize(
        segmentation_only_results, entity_path="/COCOObjectDetectionResultsWithOnlySegmentation"
    )

    # ======================= Convert Segmentation ==============================
    as_rle = bbox_segmentation_results.segmentations[0]
    as_polygon = bbox_segmentation_results.get_segmentation(0, "polygon")
    as_mask = bbox_segmentation_results.get_segmentation(0, "mask")
    logger.info(f"Segmentation 0 as RLE: {as_rle}")
    logger.info(f"Segmentation 0 as polygon: {as_polygon}")
    logger.info(f"Segmentation 0 as mask: shape={as_mask.shape}, dtype={as_mask.dtype}")

    mask_back_to_rle = datatypes.COCOObjectDetectionResults.convert_segmentation(
        as_mask, "mask", "rle"
    )
    polygon_to_mask = datatypes.COCOObjectDetectionResults.convert_segmentation(
        as_polygon,
        datatypes.SegmentationFormat.POLYGON,
        datatypes.SegmentationFormat.MASK,
        height=H,
        width=W,
    )
    logger.info(f"Mask converted back to RLE: {mask_back_to_rle}")
    logger.info(
        f"Polygon converted to mask: shape={polygon_to_mask.shape}, dtype={polygon_to_mask.dtype}"
    )

    # ======================= Serialize / Deserialize ===========================
    start = time.perf_counter()
    serialized = datatypes.serialize(bbox_results)
    serialization_ms = (time.perf_counter() - start) * 1000

    start = time.perf_counter()
    deserialized = datatypes.deserialize(serialized)["param_0"]
    deserialization_ms = (time.perf_counter() - start) * 1000

    logger.info(f"Deserialized COCOObjectDetectionResults: {deserialized}")
    logger.info(f"Round-trip successful: {deserialized == bbox_results}")
    logger.info(f"Serialization time: {serialization_ms:.3f} ms")
    logger.info(f"Deserialization time: {deserialization_ms:.3f} ms")


if __name__ == "__main__":
    coco_object_detection_results_example()