Filter Segments By Mask

SUMMARY

Filter Segments By Mask keeps only the segments of an existing label map that overlap a region-of-interest mask.

It is a post-processing step, not a standalone segmenter: it takes the labels produced by a prior segmentation call (e.g. segment_image_using_felzenszwalb or segment_image_using_slic_superpixel) together with the original image and a mask, and keeps only the segments that overlap the non-zero region of mask, discarding the rest (relabeling them as background). Use it to restrict a segmentation result to a region of interest, such as a hand-drawn ROI or the output of another detection/segmentation skill.

Use this Skill when you want to restrict a label map to only the segments that overlap a region-of-interest mask.

The Skill

from telekinesis import cornea

filtered_image = cornea.filter_segments_by_mask(
    image=image,
    labels=labels,
    mask=mask,
)

API Reference

Full parameter and return type documentation for filter_segments_by_mask.

View Reference →

Example

Input Image

Original image with superpixels

Output Image

Filtered image - only superpixels within mask region shown

The Code

"""
Demonstrates filtering superpixels based on a mask.
"""

import numpy as np
from loguru import logger
import rerun as rr
import rerun.blueprint as rrb

from telekinesis import cornea, datatypes

def filter_segments_by_mask_example():
    """Filters superpixels based on intersection with a mask."""
    # ===================== Load Image ==========================================
    image_url = "https://assets.telekinesis.ai/examples/v1/images/eggs_carton.jpg"
    image = datatypes.Image.from_url(url=image_url)

    # ===================== Run Skill ==========================================
    superpixel_segmentation_image = cornea.segment_image_using_felzenszwalb(
        image=image, scale=500, sigma=1, min_size=200
    )
    h, w, _ = image.shape
    mask = np.zeros((h, w), dtype=np.uint8)
    mask[:, : w // 3] = 255
    mask = datatypes.SegmentationImage(mask)
    filtered_image = cornea.filter_segments_by_mask(
        image=image, 
        labels=superpixel_segmentation_image, 
        mask=mask
    )

    # ===================== Log ================================================
    logger.success(f"Filtered {image} superpixels by mask.")
    logger.success(f"Results: {filtered_image}")
    logger.info(f"Filtered image label codes: {filtered_image.label_codes}")
    logger.info(f"Filtered image number of labels: {filtered_image.number_of_labels}")
    logger.info(f"Filtered image shape: {filtered_image.shape}")
    logger.info(f"Filtered image dtype: {filtered_image.dtype}")

    # ===================== Visualization  (Optional) ======================
    rr.init("filter_segments_by_mask_example", spawn=True)
    blueprint = rrb.Horizontal(
        rrb.Spatial2DView(origin="/input_image", name="Input"),
        rrb.Spatial2DView(origin="/filtering_mask", name="Mask"),
        rrb.Spatial2DView(origin="/filtered_image", name="Output"),
    )
    rr.send_blueprint(blueprint)
    datatypes.visualize(image, entity_path="/input_image")
    datatypes.visualize(mask, entity_path="/filtering_mask")
    datatypes.visualize(filtered_image, entity_path="/filtered_image")


if __name__ == "__main__":
    filter_segments_by_mask_example()

Runnable examples are available in the Telekinesis examples repository.

Follow the README in that repository to set up the environment, run this specific example with:

cd telekinesis-examples
python examples/segmentation/filter_segments_by_mask.py

Parameter Configuration

Parameter	Type	Default	Description
image	datatypes.Image | np.ndarray	required	The original image the segments were computed from, shape (H, W, 3)
labels	datatypes.SegmentationImage | np.ndarray	required	The label map to filter, typically the output of another Cornea segmentation skill, shape (H, W)
mask	datatypes.SegmentationImage | np.ndarray	required	The region of interest, shape (H, W), matching labels. Zero pixels are treated as outside the mask; any non-zero value marks a pixel to keep (a plain 0/255 binary mask works)

Returns

Type	Description
datatypes.SegmentationImage	A per-pixel label map, shape (H, W), containing only the segments that overlapped mask. Use .data for the raw label array, .label_codes for the sorted array of unique ids remaining, .number_of_labels for how many segments survived the filter, and .shape/.dtype for its size and label dtype.

Raises

Exception	Condition
TypeError	A parameter's value does not match its expected type (see the Parameter Configuration table above)
ValueError	mask and labels don't share the same (H, W)
ConfigurationError	The TELEKINESIS_API_KEY environment variable is not set
SerializationError	The request input failed to serialize, the response was not returned as an Arrow stream, or the response failed to deserialize
RequestTimeoutError	The request to the Cornea service timed out
TransportError	A network failure occurred before a response was received
ClientError	The Cornea service rejected the request due to invalid or malformed input (HTTP 400/422), an unrecognized endpoint (HTTP 404), or another unexpected 4xx response
AuthenticationError	The API key was rejected as invalid or expired (HTTP 401)
AuthenticationServiceError	The authentication service returned an invalid response, was temporarily unavailable, or timed out (HTTP 502/503/504)
ServerError	The Cornea service returned a 5xx or otherwise unexpected error response

How to Tune the Parameters

filter_segments_by_mask has no numeric threshold to tune — mask is an input defining the region of interest, not a knob with a range to sweep. The only thing you control is the shape of mask itself:

• A smaller/tighter non-zero region keeps fewer segments — only those overlapping that narrower area survive.
• A larger/looser non-zero region keeps more segments — anything overlapping any part of it survives, even segments that mostly fall outside the mask.
• mask's shape must match labels — build it, for example, from a fraction of the image (as in the example above), from another detection/segmentation skill's output, or from a hand-drawn/interactive selection.

TIP

Because any overlap with the mask is enough to keep a segment, a coarse or slightly oversized mask can still let unwanted segments through near its edges. Tighten the mask boundary if you see extra segments surviving that shouldn't.

Where to Use the Skill

Common pipelines include:

• Region-of-interest filtering – Restrict output from segment_image_using_felzenszwalb or segment_image_using_slic_superpixel to a specific area of the image
• Interactive segmentation – Keep only the segments inside a user-drawn or user-selected region
• Detection-guided filtering – Use a mask derived from another detection or segmentation skill's output to restrict segments to a known object location
• Pipeline cleanup – Narrow a superpixel map down to plausible object candidates before area or color filtering

Alternative Skills

Skill	vs. Filter Segments By Mask
filter_segments_by_area	Filters the same kind of label map by each segment's pixel count instead of spatial overlap. Use together to combine size and location constraints.
filter_segments_by_color	Filters the label map by each segment's mean pixel intensity instead of spatial overlap. Use together to combine color and location constraints.
segment_image_using_felzenszwalb	A primary segmenter that produces the labels this Skill filters. Run it first.
segment_image_using_slic_superpixel	Another primary segmenter whose output labels this Skill can filter. Run it first.

When Not to Use the Skill

Do not use Filter Segments By Mask when:

• You don't have a mask - build one first (e.g. from a fraction of the image, another detection/segmentation skill, or user input), or use filter_segments_by_area/filter_segments_by_color instead
• The distinguishing property is size, not location - use filter_segments_by_area instead
• The distinguishing property is color, not location - use filter_segments_by_color instead
• mask doesn't match the shape of labels - resize or regenerate the mask so it matches before filtering

TIP

mask need not be binary — any non-zero value is treated as "inside" the region of interest, so an existing multi-label SegmentationImage (e.g. from another segmentation skill) can be passed directly without first converting it to 0/255.