Track Detections Using McByte

SUMMARY

McByteTracker links object detections across frames and assigns each object a persistent tracker ID. It runs the upstream McByte association and mask lifecycle, with box-prompted SAM masks and CUTIE mask propagation executed locally through ONNX Runtime. No detector is included: supply bounding boxes from your own detector on every frame.

Installation

Install the base package using the Trackers setup guide, then add the mcbyte extra:

python -m pip install "telekinesis-trackers[mcbyte]"

The extra installs the upstream trackers[mask] package (pinned to 2.6.0) and supervision. Detections are passed in and returned as supervision.Detections.

CPU only

McByte's SAM and CUTIE models always use ONNX Runtime's CPU provider. The [gpu] installation and the device options used by other trackers do not apply.

Model Bundles

McByte needs two local ONNX bundles; there is no default download. Create them from a trackers source checkout, in a separate export environment:

python -m pip install -r scripts/requirements-export.txt
python scripts/prepare_bundles.py --models mcbyte --smoke-test

This exports SAM ViT-B to bundles/mcbyte/sam/vit-b and CUTIE to bundles/mcbyte/cutie/480x864-spatial. The CUTIE bundle must be exported with dynamic object counts; the exporter does this for McByte automatically. Alternatively, pass sam_model_url and cutie_model_url to download bundle ZIPs into the given directories.

The Skill

tracked = tracker.update(detections, image_rgb)

Call update() once per frame with that frame's detections:

import numpy as np
import supervision as sv
from PIL import Image
from telekinesis.trackers import McByteTracker

tracker = McByteTracker(
    "bundles/mcbyte/sam/vit-b",
    "bundles/mcbyte/cutie/480x864-spatial",
)

image_rgb = np.asarray(Image.open("frame_000.png").convert("RGB"))
# Replace with your detector's output for this frame.
detections = sv.Detections(
    xyxy=np.array([[100, 80, 220, 300], [300, 120, 420, 340]], dtype=np.float32),
    confidence=np.array([0.92, 0.85], dtype=np.float32),
    class_id=np.array([0, 0]),
)

tracked = tracker.update(detections, image_rgb)
print(tracked.tracker_id)

Boxes use (x1, y1, x2, y2) pixel coordinates in the frame passed with them.

Constructor Parameters

Parameter	Default	Description
sam_model_dir	Required	Local SAM ONNX bundle (format telekinesis-sam-onnx, ViT-B)
cutie_model_dir	Required	Local CUTIE ONNX bundle exported with dynamic object counts
sam_model_url	None	Optional SAM bundle ZIP URL, downloaded into sam_model_dir when the bundle is missing or incomplete
cutie_model_url	None	Optional CUTIE bundle ZIP URL, downloaded into cutie_model_dir when the bundle is missing or incomplete
max_internal_size	480	CUTIE short-side limit, as in CutieTracker; -1 disables reduction, otherwise at least 16
threads	1	Positive ONNX Runtime intra-operation thread count, used by both models
**options		Forwarded to the upstream trackers McByte tracker

All parameters after the two bundle directories are keyword-only. mask_manager, mask_config, and enable_mask_manager cannot be passed in options, because this wrapper supplies its own ONNX-backed mask manager. See the upstream trackers documentation for the remaining association options.

Methods

Method	Behavior
update(detections, image_rgb)	Associate this frame's supervision.Detections with existing tracks and return detections with persistent tracker IDs
reset_session()	Clear all tracks and mask state

image_rgb must be a uint8 RGB array (H, W, 3). Internally, SAM prompts each tracked box on the frame scaled to 1024 pixels on its longer side, and its masks are resized back to the frame's resolution before CUTIE propagates them.

McByte uses a bounded CUTIE memory policy from this package rather than upstream McByte's optional long-term memory compression, so tracking behavior can differ from the reference implementation.

Common Errors

• Passing anything other than supervision.Detections raises TypeError.
• Non-finite boxes, boxes with negative width or height, or confidences outside [0, 1] raise ValueError.
• A CUTIE bundle without dynamic object counts, an unsupported SAM bundle, or a missing bundle without a download URL is rejected on construction.
• A missing mcbyte extra raises ImportError with the installation command.

Call reset_session() before processing an unrelated video.