Interactive Segmentation with SAM3¶

This notebook demonstrates how to segment remote sensing images interactively using the Segment Anything Model 3 (SAM3).

Installation¶

First, make sure you have the required dependencies installed:

In [ ]:

# %pip install "segment-geospatial[samgeo3]"

# %pip install "segment-geospatial[samgeo3]"

In [ ]:

# %pip install git+https://github.com/huggingface/transformers.git

# %pip install git+https://github.com/huggingface/transformers.git

Import Libraries¶

In [ ]:

import leafmap
from samgeo import SamGeo3, download_file

import leafmap from samgeo import SamGeo3, download_file

Download Sample Data¶

Let's download a sample satellite image covering the University of California, Berkeley, for testing:

In [ ]:

url = "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/uc_berkeley.tif"
image_path = download_file(url)

url = "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/uc_berkeley.tif" image_path = download_file(url)

In [ ]:

m = leafmap.Map()
m.add_raster(image_path, layer_name="Satellite image")
m

m = leafmap.Map() m.add_raster(image_path, layer_name="Satellite image") m

Request access to SAM3¶

To use SAM3, you need to request access by filling out this form on Hugging Face: https://huggingface.co/facebook/sam3

Once you have access, uncomment the following code block and run it.

In [ ]:

# from huggingface_hub import login
# login()

# from huggingface_hub import login # login()

Initialize SAM3¶

When initializing SAM3, you can choose the backend from "meta", or "transformers".

In [ ]:

sam3 = SamGeo3(
    backend="transformers", device=None, checkpoint_path=None, load_from_HF=True
)

sam3 = SamGeo3( backend="transformers", device=None, checkpoint_path=None, load_from_HF=True )

Set the image¶

You can set the image by either passing the image path or the image URL.

In [ ]:

sam3.set_image(image_path)

sam3.set_image(image_path)

Generate masks with text prompt¶

In [ ]:

sam3.generate_masks(prompt="building")

sam3.generate_masks(prompt="building")

In [ ]:

sam3.save_masks("masks.tif")

sam3.save_masks("masks.tif")

Show the results¶

In [ ]:

sam3.show_anns()

sam3.show_anns()

In [ ]:

sam3.show_masks()

sam3.show_masks()

Interactive segmentation¶

Enter a text prompt or draw a rectangle on the map, then click on the Segment button to perform instance segmentation.

In [ ]:

sam3.show_map(height="700px", min_size=10)

sam3.show_map(height="700px", min_size=10)

Segmentation by text prompts.

Segmentation by bounding boxes.