Run Inference

SUMMARY

Use one local inference workflow for every exported Iris model. The artifact extension or optional model name selects the RFDETR or SAM3-LoRA runtime.

Inference Workflow

Export the model first with Export a Model. Local inference uses telekinesis.iris.deploy.Model directly; it does not upload or register the model with a backend.

Create inference.py

"""Run local inference with an exported Iris model."""

import argparse
from pathlib import Path

import rerun as rr
from loguru import logger

from telekinesis.iris import deploy, visualization


def inference_model_example(args: argparse.Namespace) -> None:
    """Load an exported model and run inference on one image."""

    # ===================== Load Model =====================================
    model = deploy.Model(
        args.model,
        model_name=args.model_name,
        class_names=args.class_names,
        num_select=args.max_detections,
        device=args.device,
    )

    # ===================== Run Skill ======================================
    prediction = model.predict(
        image=args.image,
        confidence_threshold=args.confidence_threshold,
        prompts=args.prompts,
    )
    logger.info(f"Found {len(prediction)} detections in {args.image.name}")
    for box, score, class_id, class_name in zip(
        prediction.boxes,
        prediction.scores,
        prediction.class_ids,
        prediction.class_names or map(str, prediction.class_ids),
        strict=True,
    ):
        logger.info(
            f"class={class_name} id={class_id} score={score:.3f} "
            f"box={box.round(1).tolist()}"
        )

    # ===================== Visualization =================================
    rr.init("iris_inference", spawn=True)
    visualization.visualize_prediction(args.image, prediction)


if __name__ == "__main__":
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--model", type=Path, required=True, help="Path to an exported ONNX model or Iris bundle.")
    parser.add_argument("--image", type=Path, required=True, help="Path to the image on which to run inference.")
    parser.add_argument("--model-name", help="Optional model name; the artifact extension selects the family when omitted.")
    parser.add_argument("--class-names", nargs="*", help="Optional RFDETR class names or SAM3-LoRA prompt names.")
    parser.add_argument("--prompts", nargs="*", help="Optional prompts for a SAM3-LoRA model.")
    parser.add_argument("--confidence-threshold", type=float, help="Optional per-inference confidence threshold.")
    parser.add_argument("--max-detections", type=int, help="Maximum detections to return.")
    parser.add_argument("--device", help="PyTorch device for SAM3-LoRA, such as 'cuda' or 'cpu'.")
    inference_model_example(parser.parse_args())

Run Inference

Run an RFDETR ONNX model. Specify class names in training-category order:

python inference.py \
  --model artifacts/rfdetr/model.onnx \
  --model-name seg-medium \
  --image data/test-image.png \
  --class-names green_circle red_triangle blue_square

Run a SAM3-LoRA bundle. The family can be selected from the .pt artifact extension or explicitly with --model-name:

python inference.py \
  --model artifacts/sam3-lora/model.pt \
  --model-name sam3-lora \
  --image data/test-image.png \
  --prompts green_circle red_triangle blue_square \
  --device cuda

Match Labels and Prompts

For RFDETR, keep --class-names in the same order as the training dataset categories. Use --prompts only with SAM3-LoRA; supplying prompts to RFDETR raises an error.

Run the Example

Runnable versions are available in the Telekinesis examples repository:

cd telekinesis-examples
python examples/iris/infer_custom_rfdetr_model.py \
  --model artifacts/rfdetr/model.onnx \
  --model-name seg-medium \
  --image data/test-image.png \
  --class-names green_circle red_triangle blue_square

python examples/iris/infer_custom_sam3_lora_model.py \
  --model artifacts/sam3-lora/model.pt \
  --model-name sam3-lora \
  --image data/test-image.png \
  --prompts green_circle red_triangle blue_square \
  --device cuda

Model Reference

Model Constructor Parameters

Parameter	Default	Description
model_path	required	Path to an RFDETR ONNX artifact or SAM3-LoRA Iris bundle
model_name	None	Optional public model name; otherwise inferred from the artifact extension
class_names	None	Optional category-name mapping or sequence; also provides default SAM3 prompt names
postprocess	None	Family-specific RFDETRPostprocessConfig or SAM3PostprocessConfig
num_select	None	Deprecated shorthand for the maximum number of detections; use postprocess.max_detections instead
background_class_id	0	RFDETR class slot excluded from predictions; set to None to disable filtering
providers	None	Ordered ONNX Runtime execution providers for RFDETR
input_size	None	Spatial input size for a dynamic RFDETR ONNX model
array_color_format	"rgb"	Color order for NumPy image inputs: "rgb" or "bgr"
device	None	PyTorch device used for SAM3-LoRA inference

Methods

Method	Returns	Description
model.predict(image, confidence_threshold=None, prompts=None)	Prediction	Runs inference on an image path or HWC NumPy array. prompts is supported only for SAM3-LoRA.
model(image)	Prediction	Runs inference with the configured default post-processing settings.

Post-processing Configuration

Pass a family-specific postprocess configuration to deploy.Model when the command-line threshold and detection-limit options are not sufficient.

RFDETRPostprocessConfig

Option	Default	Description
confidence_threshold	0.3	Minimum retained confidence in the range [0, 1]
max_detections	300	Maximum predictions returned per image

SAM3PostprocessConfig

Option	Default	Description
confidence_threshold	0.3	Minimum retained confidence in the range [0, 1]
max_detections	300	Maximum predictions returned per image
per_prompt_nms_threshold	0.5	Per-prompt NMS IoU threshold; None disables NMS
prompt_conflict_policy	"multi_label"	"multi_label" retains overlaps; "best_prompt" resolves them
cross_prompt_iou_threshold	0.8	IoU used to identify cross-prompt conflicts

Next Steps

Export a Model

Create an ONNX model or Iris bundle from a training checkpoint.

Export a model →

Configuration

Review model and post-processing settings.

View configuration →