SAM3 Image Segmentation for JPG images¶

This notebook demonstrates how to use the Segment Anything Model 3 (SAM3) for segmenting JPG images using the samgeo3 module.

Installation¶

First, make sure you have the required dependencies installed:

In [ ]:

# %pip install "segment-geospatial[samgeo3]" git+https://github.com/huggingface/transformers.git

# %pip install "segment-geospatial[samgeo3]" git+https://github.com/huggingface/transformers.git

Import Libraries¶

In [ ]:

from samgeo import SamGeo3, download_file
from samgeo.common import show_image

from samgeo import SamGeo3, download_file from samgeo.common import show_image

Download Sample Data¶

Let's download a sample satellite image covering the University of California, Berkeley, for testing:

In [ ]:

url = "https://raw.githubusercontent.com/facebookresearch/sam3/refs/heads/main/assets/images/test_image.jpg"
image_path = download_file(url)

url = "https://raw.githubusercontent.com/facebookresearch/sam3/refs/heads/main/assets/images/test_image.jpg" image_path = download_file(url)

In [ ]:

show_image(image_path)

show_image(image_path)

Request access to SAM3¶

To use SAM3, you need to request access by filling out this form on Hugging Face: https://huggingface.co/facebook/sam3

Once you have access, uncomment the following code block and run it.

In [ ]:

# from huggingface_hub import login
# login()

# from huggingface_hub import login # login()

Initialize SAM3¶

When initializing SAM3, you can choose the backend from "meta", or "transformers".

In [ ]:

sam3 = SamGeo3(
    backend="transformers", device=None, checkpoint_path=None, load_from_HF=True
)

sam3 = SamGeo3( backend="transformers", device=None, checkpoint_path=None, load_from_HF=True )

Set the image¶

You can set the image by either passing the image path or the image URL.

In [ ]:

sam3.set_image(image_path)

sam3.set_image(image_path)

Generate masks with text prompt¶

In [ ]:

sam3.generate_masks(prompt="person")

sam3.generate_masks(prompt="person")

Show the results¶

In [ ]:

sam3.show_anns()

sam3.show_anns()

In [ ]:

sam3.show_masks()

sam3.show_masks()

Generate masks by bounding boxes¶

In [ ]:

# Define boxes in [xmin, ymin, xmax, ymax] format
boxes = [[480, 290, 590, 650]]

# Optional: specify which boxes are positive/negative prompts
box_labels = [True]  # True=include, False=exclude

# Generate masks
sam3.generate_masks_by_boxes(boxes, box_labels)

# Define boxes in [xmin, ymin, xmax, ymax] format boxes = [[480, 290, 590, 650]] # Optional: specify which boxes are positive/negative prompts box_labels = [True] # True=include, False=exclude # Generate masks sam3.generate_masks_by_boxes(boxes, box_labels)

In [ ]:

sam3.show_boxes(boxes, box_labels)

sam3.show_boxes(boxes, box_labels)

Show the results¶

In [ ]:

sam3.show_anns()

sam3.show_anns()

Save Masks¶

Save the generated masks to a file. If the input is a GeoTIFF, the output will be a GeoTIFF with the same georeferencing. Otherwise, it will be saved as PNG.

In [ ]:

# Save as binary mask (all foreground pixels are 255)
sam3.save_masks(output="person_masks_binary.png", unique=False)

# Save as binary mask (all foreground pixels are 255) sam3.save_masks(output="person_masks_binary.png", unique=False)

In [ ]:

sam3.show_masks(cmap="coolwarm")

sam3.show_masks(cmap="coolwarm")