Video Segmentation and Object Tracking with SAM 3¶

This notebook demonstrates how to use SAM 3 for video segmentation and object tracking.

Installation¶

SAM 3 requires CUDA-capable GPU. Install with:

In [ ]:

# %pip install "segment-geospatial[samgeo3]"

# %pip install "segment-geospatial[samgeo3]"

Import Libraries¶

In [ ]:

import os
from samgeo import SamGeo3Video, download_file

import os from samgeo import SamGeo3Video, download_file

Initialize Video Predictor¶

The SamGeo3Video class provides a simplified API for video segmentation. It automatically uses all available GPUs.

In [ ]:

sam = SamGeo3Video()

sam = SamGeo3Video()

Load a Video¶

You can load from different sources:

• MP4 video file
• Directory of JPEG frames
• Directory of GeoTIFFs (for remote sensing time series)

In [ ]:

url = "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/basketball.mp4"
video_path = download_file(url)

url = "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/basketball.mp4" video_path = download_file(url)

In [ ]:

sam.set_video(video_path)

sam.set_video(video_path)

In [ ]:

sam.show_video(video_path)

sam.show_video(video_path)

Text-Prompted Segmentation¶

Use natural language to describe objects. SAM 3 finds all instances and tracks them.

In [ ]:

# Segment all players in the video
sam.generate_masks("player")

# Segment all players in the video sam.generate_masks("player")

Visualize Results¶

Customize player names:

In [ ]:

player_names = {}
for i in range(15):
    player_names[i] = f"Player {i}"
sam.show_frame(0, axis="on", show_ids=player_names)

player_names = {} for i in range(15): player_names[i] = f"Player {i}" sam.show_frame(0, axis="on", show_ids=player_names)

Remove objects¶

In [ ]:

# Remove objects and re-propagate
sam.remove_object(obj_id=[5, 8, 12, 13])
sam.propagate()
sam.show_frame(0, show_ids=player_names)

# Remove objects and re-propagate sam.remove_object(obj_id=[5, 8, 12, 13]) sam.propagate() sam.show_frame(0, show_ids=player_names)

Save Results¶

Save masks as images or create an output video.

In [ ]:

os.makedirs("output", exist_ok=True)

# Save mask images
sam.save_masks("output/masks")

os.makedirs("output", exist_ok=True) # Save mask images sam.save_masks("output/masks")

In [ ]:

# Save video with blended masks
sam.save_video("output/players_segmented.mp4", fps=60, show_ids=player_names)

# Save video with blended masks sam.save_video("output/players_segmented.mp4", fps=60, show_ids=player_names)

In [ ]:

sam.show_video("output/players_segmented.mp4")

sam.show_video("output/players_segmented.mp4")

Close Session¶

Close the session to free GPU resources.

In [ ]:

sam.close()

sam.close()

To completely shutdown and free all resources:

In [ ]:

sam.shutdown()

sam.shutdown()