Segmenting remote sensing imagery with box prompts and SAM 3¶
This notebook shows how to generate object masks from box prompts with the Segment Anything Model 3 (SAM 3).
Make sure you use GPU runtime for this notebook. For Google Colab, go to Runtime -> Change runtime type and select GPU as the hardware accelerator.
Install dependencies¶
Uncomment and run the following cell to install the required dependencies.
# %pip install "segment-geospatial[samgeo3]"
Import libraries¶
import leafmap
from samgeo import SamGeo3, download_file
from samgeo.common import raster_to_vector, regularize
Download Sample Data¶
Let's download a sample satellite image covering Washington State:
image_url = "https://github.com/opengeos/datasets/releases/download/places/wa_building_image.tif"
image = download_file(image_url)
m = leafmap.Map()
m.add_raster(image, layer_name="Satellite image")
m
Initialize SAM 3¶
To use point and box prompts (SAM1-style interactive segmentation), initialize SAM3 with enable_inst_interactivity=True.
sam = SamGeo3(backend="meta", enable_inst_interactivity=True)
Specify the image to segment.
sam.set_image(image)
Create bounding boxes¶
Use the drawing tools to draw some rectangles around the features you want to extract, such as trees, buildings.
If no rectangles are drawn, the default bounding boxes will be used as follows:
if m.user_rois is not None:
boxes = m.user_rois
else:
boxes = [
[-117.5995, 47.6518, -117.5988, 47.652],
[-117.5987, 47.6518, -117.5979, 47.652],
]
Segment the image¶
Use the generate_masks_by_boxes_inst() method to segment the image with specified bounding boxes. The boxes parameter accepts a list of bounding box coordinates in the format of [[xmin, ymin, xmax, ymax], [xmin, ymin, xmax, ymax], ...].
sam.generate_masks_by_boxes_inst(boxes=boxes, box_crs="EPSG:4326")
Save the masks to a file.
sam.save_masks(output="mask.tif", dtype="uint8")
Display the result¶
Add the segmented image to the map.
m.add_raster("mask.tif", cmap="viridis", nodata=0, opacity=0.5, layer_name="Mask")
m
Use an existing vector file as box prompts¶
Alternatively, you can specify a file path to a vector file. Let's download a sample vector file from GitHub.
url = "https://github.com/opengeos/datasets/releases/download/samgeo/building_bboxes.geojson"
geojson = "building_bboxes.geojson"
download_file(url, geojson)
Display the vector data on the map.
m = leafmap.Map()
m.add_raster(image, layer_name="Image")
style = {
"color": "#ffff00",
"weight": 2,
"fillColor": "#7c4185",
"fillOpacity": 0,
}
m.add_vector(geojson, style=style, zoom_to_layer=True, layer_name="Bboxes")
m
Segment image with box prompts from vector file¶
The generate_masks_by_boxes_inst() method can directly accept a file path to a vector file (GeoJSON, Shapefile, etc.). It will automatically extract bounding boxes from geometries and filter out any boxes outside the image bounds.
output_masks = "building_masks.tif"
sam.generate_masks_by_boxes_inst(
boxes=geojson,
box_crs="EPSG:4326",
output=output_masks,
dtype="uint16",
multimask_output=False,
)
Display the segmented masks on the map.
m.add_raster(
output_masks, cmap="jet", nodata=0, opacity=0.5, layer_name="Building masks"
)
m
Convert raster to vector¶
output_vector = "building_vector.geojson"
raster_to_vector(output_masks, output_vector)
Regularize building footprints¶
output_regularized = "building_regularized.geojson"
regularize(output_vector, output_regularized)
m.add_vector(
output_regularized, style=style, layer_name="Building regularized", info_mode=None
)