Video Segmentation with Point Prompts and SAM 3¶

This notebook demonstrates how to use SAM 3 for video segmentation and tracking. You will learn how to install the required libraries, load videos from different sources, apply point prompts for segmentation, and visualize the results step by step.

Installation¶

SAM 3 requires CUDA-capable GPU. Install with:

In [ ]:

# %pip install "segment-geospatial[samgeo3]"

# %pip install "segment-geospatial[samgeo3]"

Import Libraries¶

In [ ]:

import os
from samgeo import SamGeo3Video, download_file

import os from samgeo import SamGeo3Video, download_file

Initialize Video Predictor¶

The SamGeo3Video class provides a simplified API for video segmentation. It automatically uses all available GPUs.

In [ ]:

sam = SamGeo3Video()

sam = SamGeo3Video()

Load a Video¶

You can load from different sources:

• MP4 video file
• Directory of JPEG frames
• Directory of GeoTIFFs (for remote sensing time series)

In [ ]:

url = "https://github.com/opengeos/datasets/releases/download/videos/cars.mp4"
video_path = download_file(url)

url = "https://github.com/opengeos/datasets/releases/download/videos/cars.mp4" video_path = download_file(url)

In [ ]:

sam.set_video(video_path)

sam.set_video(video_path)

In [ ]:

sam.show_video(video_path)

sam.show_video(video_path)

Point-Prompted Segmentation¶

First, let's initialize the tracker.

In [ ]:

sam.init_tracker()

sam.init_tracker()

In [ ]:

sam.show_frame(0, axis="on")

sam.show_frame(0, axis="on")

Add point prompts¶

In [ ]:

sam.add_point_prompts([[300, 200]], [1], obj_id=1, frame_idx=0)

sam.add_point_prompts([[300, 200]], [1], obj_id=1, frame_idx=0)

In [ ]:

sam.add_point_prompts([[420, 200]], [1], obj_id=2, frame_idx=0)

sam.add_point_prompts([[420, 200]], [1], obj_id=2, frame_idx=0)

In [ ]:

sam.propagate()

sam.propagate()

Visualize Results¶

In [ ]:

# Show the first frame with masks
sam.show_frame(0, axis="on")

# Show the first frame with masks sam.show_frame(0, axis="on")

In [ ]:

# Show multiple frames in a grid
sam.show_frames(frame_stride=20, ncols=3)

# Show multiple frames in a grid sam.show_frames(frame_stride=20, ncols=3)

Remove Objects¶

Remove specific objects by ID and re-propagate.

In [ ]:

# Remove object 2 and re-propagate
sam.remove_object(2)
sam.propagate()
sam.show_frame(0)

# Remove object 2 and re-propagate sam.remove_object(2) sam.propagate() sam.show_frame(0)

Refine with Multiple Points¶

Use positive and negative points to refine the mask.

In [ ]:

# Refine to segment only the windshield (not the rest of the car)
sam.add_point_prompts(
    points=[[335, 195], [335, 220]],  # positive: windshield, negative: car body
    labels=[1, 0],  # positive, negative
    obj_id=1,
    frame_idx=0,
)
sam.propagate()
sam.show_frames(frame_stride=20, ncols=3)

# Refine to segment only the windshield (not the rest of the car) sam.add_point_prompts( points=[[335, 195], [335, 220]], # positive: windshield, negative: car body labels=[1, 0], # positive, negative obj_id=1, frame_idx=0, ) sam.propagate() sam.show_frames(frame_stride=20, ncols=3)

Save Results¶

Save masks as images or create an output video.

In [ ]:

os.makedirs("output", exist_ok=True)

# Save mask images
sam.save_masks("output/masks")

os.makedirs("output", exist_ok=True) # Save mask images sam.save_masks("output/masks")

In [ ]:

# Save video with blended masks
sam.save_video("output/segmented.mp4", fps=25)

# Save video with blended masks sam.save_video("output/segmented.mp4", fps=25)

Close Session¶

Close the session to free GPU resources.

In [ ]:

sam.close()

sam.close()

To completely shutdown and free all resources:

In [ ]:

sam.shutdown()

sam.shutdown()