Video and Live Examples
These commands use the examples/ scripts in a trackers source checkout. Run them from that repository's root. Installing the package alone does not provide these script paths.
The scripts cover CUTIE, SAM3, RITM, TAPIR, and TAPNext++. For McByte and M3T pose tracking, use the Python APIs in their guides; these command-line scripts do not support them.
Setup
Use Python 3.11 or newer and install the checkout with its example dependencies:
python -m pip install ".[examples]"This adds OpenCV and Loguru. Interactive selection requires a graphical desktop; live tracking also requires an accessible camera. The commands below select CPU explicitly for the tracker. For CUDA, follow the GPU setup instructions after installation and use --device cuda.
Track Masks Through a Video
Prepare one binary mask image per object on the video's first frame. Nonzero pixels identify that object. This example uses a 640 by 480 video and masks:
python examples/track_video.py --tracker cutie --video clip.mp4 --masks object_a.png object_b.png --resize 640 480 --device cpu --out tracked.mp4 --save-masksEach mask must be exactly the selected height and width. IDs are assigned in command-line order, starting at 1; later masks overwrite earlier ones at overlapping pixels. The script resizes video frames to --resize, so choose dimensions with the intended aspect ratio and prepare masks for that resized frame.
--out selects the overlay video. Omit it to write <input_stem>_tracked.mp4. --save-masks also writes per-frame label PNGs. Use --model-dir to supply a local CUTIE bundle instead of downloading the default.
Always pass --tracker: this script defaults to sam3, which requires --model-dir.
Live Mask Tracking
Start CUTIE on camera zero:
python examples/track_live.py --tracker cutie --camera 0 --device cpu --out live_tracked.mp4On the initial frame, left-drag to paint an object, right-drag to erase, and press Enter to start tracking. During tracking, press c to pause and correct the current mask with the same controls, then press Enter to resume; the script passes the edited mask to the tracker's correct(). Press q to stop.
| Key | Action during initialization |
|---|---|
1 through 9 | Select an object ID |
n / p | Select next / previous object ID |
[ / ] | Shrink / grow the painting brush |
u / c | Undo / clear the selected object |
Enter or s | Accept the masks and seed tracking |
| Escape | Cancel initialization |
To initialize with RITM clicks instead of painting:
python examples/track_live.py --tracker cutie --camera 0 --ritm --device cpuLeft-click inside the selected object and right-click on unwanted regions. Switch object IDs to initialize more objects, then press Enter. --initializer-dir supplies a local RITM bundle and also enables click initialization. The live script constructs RITM with its default device="auto"; --device controls CUTIE only. To select RITM's device explicitly, use the Python initializer API.
With the default dynamic CUTIE bundle, this script preserves camera resolution and defaults to --max-internal-size -1. The CutieTracker Python constructor instead defaults to 480. Set --max-internal-size 480 to reduce internal computation, or --resize WIDTH HEIGHT to resize the incoming frames. Live resizing defaults to --resize-mode contain, preserving aspect ratio with padding; cover crops, and stretch changes aspect ratio.
SAM3 Mask Tracking
Both mask scripts accept --tracker sam3 with a local SAM3 bundle. SAM3 bundles have fixed dimensions, so --resize must match the bundle's manifest width and height:
python examples/track_video.py --tracker sam3 --model-dir bundles/sam3/default --video clip.mp4 --masks object_a.png object_b.png --resize 1024 1024 --device cpuReplace 1024 1024 with your bundle's dimensions. In the live script, --model-dir defaults to bundles/sam3/default in the checkout, and frames are resized to the bundle dimensions unless --resize is given:
python examples/track_live.py --tracker sam3 --model-dir bundles/sam3/default --camera 0 --device cpuPainting and RITM initialization work as for CUTIE. The number of objects you can paint is limited to the bundle's objects capacity. The CUTIE memory options, such as --max-internal-size, do not apply to SAM3.
Interactive Point Tracking
Select two points for the default online TAPIR bundle:
python examples/track_points.py --tracker tapir --video clip.mp4 --device cpu --out tracked_points.mp4Left-click to add points, right-click or press u to undo, then press Enter. Use --query-frame 10 to select points on a later frame; causal tracking starts there and does not produce predictions for earlier frames.
Supply coordinates to skip the picker:
python examples/track_points.py --tracker tapir --video clip.mp4 --points 100,80 240,160 --device cpu --out tracked_points.mp4Coordinates must lie inside the source video. The script writes both an overlay .mp4 and a compressed .npz with xy, visible, visibility, occlusion, and frame_indices. The default TAPIR online bundle requires exactly two points; the default offline bundle also limits the clip to two frames. See TAPIR for the offline Python API.
TAPNext++ Point Tracking
Supply an existing ONNX bundle:
python examples/track_points.py --tracker tapnextpp --model-dir bundles/tapnextpp/512-dynamic --video clip.mp4 --device cpu --out tapnextpp_points.mp4Or use a checkpoint after installing its backend requirements:
python examples/track_points.py --tracker tapnextpp --checkpoint tapnextpp_512.ckpt --video clip.mp4 --device cuda --out tapnextpp_points.mp4Select one or more points with the picker for a dynamic bundle or checkpoint, or pass --points. The number remains fixed during that tracking session. The point script loads the complete input video into memory even when inference is causal; use the Python seed() / step() API for a stream processed frame by frame.