SAM3 Image Segmentation for Remote Sensing¶

This notebook demonstrates how to use the Segment Anything Model 3 (SAM3) for segmenting remote sensing images using the samgeo3 module.

Installation¶

First, make sure you have the required dependencies installed:

In [ ]:

# %pip install "segment-geospatial[samgeo3]"

# %pip install "segment-geospatial[samgeo3]"

Import Libraries¶

In [ ]:

import leafmap
from samgeo import SamGeo3, download_file

import leafmap from samgeo import SamGeo3, download_file

Download Sample Data¶

Let's download a sample satellite image covering the University of California, Berkeley, for testing:

In [ ]:

url = "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/uc_berkeley.tif"
image_path = download_file(url)

url = "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/uc_berkeley.tif" image_path = download_file(url)

In [ ]:

m = leafmap.Map()
m.add_raster(image_path, layer_name="Satellite image")
m

m = leafmap.Map() m.add_raster(image_path, layer_name="Satellite image") m

Request access to SAM3¶

To use SAM3, you need to request access by filling out this form on Hugging Face: https://huggingface.co/facebook/sam3

Once you have access, uncomment the following code block and run it.

In [ ]:

# from huggingface_hub import login
# login()

# from huggingface_hub import login # login()

Initialize SAM3¶

When initializing SAM3, you can choose the backend from "meta", or "transformers".

In [ ]:

sam3 = SamGeo3(backend="meta", device=None, checkpoint_path=None, load_from_HF=True)

sam3 = SamGeo3(backend="meta", device=None, checkpoint_path=None, load_from_HF=True)

Set the image¶

You can set the image by either passing the image path or the image URL.

In [ ]:

sam3.set_image(image_path)

sam3.set_image(image_path)

Generate masks with text prompt¶

In [ ]:

sam3.generate_masks(prompt="building")

sam3.generate_masks(prompt="building")

Show the results¶

In [ ]:

sam3.show_anns()

sam3.show_anns()

In [ ]:

sam3.show_masks()

sam3.show_masks()

Generate masks by bounding boxes¶

In [ ]:

# Define boxes in [xmin, ymin, xmax, ymax] format
boxes = [[-122.2597, 37.8709, -122.2587, 37.8717]]

# Optional: specify which boxes are positive/negative prompts
box_labels = [True]  # True=include, False=exclude

# Generate masks
sam3.generate_masks_by_boxes(boxes, box_labels, box_crs="EPSG:4326")

# Define boxes in [xmin, ymin, xmax, ymax] format boxes = [[-122.2597, 37.8709, -122.2587, 37.8717]] # Optional: specify which boxes are positive/negative prompts box_labels = [True] # True=include, False=exclude # Generate masks sam3.generate_masks_by_boxes(boxes, box_labels, box_crs="EPSG:4326")

In [ ]:

sam3.show_boxes(boxes, box_labels, box_crs="EPSG:4326")

sam3.show_boxes(boxes, box_labels, box_crs="EPSG:4326")

Show the results¶

In [ ]:

sam3.show_anns()

sam3.show_anns()

Save Masks¶

Save the generated masks to a file. If the input is a GeoTIFF, the output will be a GeoTIFF with the same georeferencing. Otherwise, it will be saved as PNG.

In [ ]:

# Save masks with unique values for each object
# Since satellite.tif is a GeoTIFF, the output will also be a GeoTIFF
sam3.save_masks(output="building_masks.tif", unique=True)

# Save masks with unique values for each object # Since satellite.tif is a GeoTIFF, the output will also be a GeoTIFF sam3.save_masks(output="building_masks.tif", unique=True)

In [ ]:

# Save as binary mask (all foreground pixels are 255)
sam3.save_masks(output="building_masks_binary.tif", unique=False)

# Save as binary mask (all foreground pixels are 255) sam3.save_masks(output="building_masks_binary.tif", unique=False)

Save Masks with Confidence Scores¶

You can also save the confidence scores for each mask. The scores indicate the model's confidence for each predicted mask.

In [ ]:

# Save masks and confidence scores
# Each pixel in the scores image will have the confidence value of its mask
sam3.save_masks(
    output="building_masks_with_scores.tif",
    save_scores="building_scores.tif",
    unique=True,
)

# Save masks and confidence scores # Each pixel in the scores image will have the confidence value of its mask sam3.save_masks( output="building_masks_with_scores.tif", save_scores="building_scores.tif", unique=True, )

In [ ]:

sam3.show_masks(cmap="coolwarm")

sam3.show_masks(cmap="coolwarm")

Visualize Confidence Scores¶

Let's visualize the confidence scores to see which masks have higher confidence:

In [ ]:

m.add_raster("building_masks.tif", layer_name="Building masks", visible=False)
m.add_raster(
    "building_scores.tif",
    layer_name="Building scores",
    cmap="coolwarm",
    opacity=0.8,
    nodata=0,
)
m

m.add_raster("building_masks.tif", layer_name="Building masks", visible=False) m.add_raster( "building_scores.tif", layer_name="Building scores", cmap="coolwarm", opacity=0.8, nodata=0, ) m