Segmenting remote sensing imagery with point prompts and SAM 3¶

This notebook shows how to generate object masks from point prompts with the Segment Anything Model 3 (SAM 3).

Make sure you use GPU runtime for this notebook. For Google Colab, go to Runtime -> Change runtime type and select GPU as the hardware accelerator.

Install dependencies¶

Uncomment and run the following cell to install the required dependencies.

In [ ]:

# %pip install "segment-geospatial[samgeo3]"

# %pip install "segment-geospatial[samgeo3]"

Import libraries¶

In [ ]:

import leafmap
from samgeo import SamGeo3, download_file

import leafmap from samgeo import SamGeo3, download_file

Download Sample Data¶

Let's download a sample satellite image covering Washington State:

In [ ]:

image_url = "https://github.com/opengeos/datasets/releases/download/places/wa_building_image.tif"
image_path = download_file(image_url)
geojson_url = "https://github.com/opengeos/datasets/releases/download/places/wa_building_centroids.geojson"
geojson_path = download_file(geojson_url)

image_url = "https://github.com/opengeos/datasets/releases/download/places/wa_building_image.tif" image_path = download_file(image_url) geojson_url = "https://github.com/opengeos/datasets/releases/download/places/wa_building_centroids.geojson" geojson_path = download_file(geojson_url)

In [ ]:

m = leafmap.Map()
m.add_raster(image_path, layer_name="Satellite image")
m

m = leafmap.Map() m.add_raster(image_path, layer_name="Satellite image") m

Initialize SAM 3¶

To use point and box prompts (SAM1-style interactive segmentation), initialize SAM3 with enable_inst_interactivity=True.

In [ ]:

sam = SamGeo3(backend="meta", enable_inst_interactivity=True)

sam = SamGeo3(backend="meta", enable_inst_interactivity=True)

Specify the image to segment.

In [ ]:

sam.set_image(image_path)

sam.set_image(image_path)

Segment the image¶

Use the generate_masks_by_points_patch() method to segment the image with specified point coordinates. You can use the draw tools to add place markers on the map. If no point is added, the default sample points will be used.

In [ ]:

if m.user_rois is not None:
    point_coords_batch = m.user_rois
else:
    point_coords_batch = [
        [-117.599896, 47.655345],
        [-117.59992, 47.655167],
        [-117.599928, 47.654974],
        [-117.599518, 47.655337],
    ]

if m.user_rois is not None: point_coords_batch = m.user_rois else: point_coords_batch = [ [-117.599896, 47.655345], [-117.59992, 47.655167], [-117.599928, 47.654974], [-117.599518, 47.655337], ]

In [ ]:

point_coords_batch

point_coords_batch

Segment the objects using the point prompts and save the output masks.

In [ ]:

sam.generate_masks_by_points_patch(
    point_coords_batch=point_coords_batch,
    point_crs="EPSG:4326",
    output="masks.tif",
    dtype="uint8",
)

sam.generate_masks_by_points_patch( point_coords_batch=point_coords_batch, point_crs="EPSG:4326", output="masks.tif", dtype="uint8", )

In [ ]:

sam.show_points(point_coords_batch, point_crs="EPSG:4326")

sam.show_points(point_coords_batch, point_crs="EPSG:4326")

In [ ]:

m.add_raster("masks.tif", cmap="viridis", nodata=0, opacity=0.7, layer_name="Mask")
m

m.add_raster("masks.tif", cmap="viridis", nodata=0, opacity=0.7, layer_name="Mask") m

Segment image with a vector dataset¶

Alternatively, you can specify a file path or HTTP URL to a vector dataset containing point geometries.

In [ ]:

m = leafmap.Map()
m.add_raster(image_path, layer_name="Image")
m.add_circle_markers_from_xy(
    geojson_path, radius=3, color="red", fill_color="yellow", fill_opacity=0.8
)
m

m = leafmap.Map() m.add_raster(image_path, layer_name="Image") m.add_circle_markers_from_xy( geojson_path, radius=3, color="red", fill_color="yellow", fill_opacity=0.8 ) m

In [ ]:

output_masks = "building_masks.tif"

output_masks = "building_masks.tif"

In [ ]:

sam.generate_masks_by_points_patch(
    point_coords_batch=geojson_path,
    point_crs="EPSG:4326",
    output=output_masks,
    dtype="uint16",
)

sam.generate_masks_by_points_patch( point_coords_batch=geojson_path, point_crs="EPSG:4326", output=output_masks, dtype="uint16", )

In [ ]:

m.add_raster(
    output_masks, cmap="jet", nodata=0, opacity=0.7, layer_name="Building masks"
)
m

m.add_raster( output_masks, cmap="jet", nodata=0, opacity=0.7, layer_name="Building masks" ) m

Interactive Segmentation¶

In [ ]:

sam.show_map(prompt="point")

sam.show_map(prompt="point")