Segmenting remote sensing imagery with box prompts¶
This notebook shows how to generate object masks from box prompts with the Segment Anything Model 2 (SAM 2).
Make sure you use GPU runtime for this notebook. For Google Colab, go to Runtime -> Change runtime type and select GPU as the hardware accelerator.
Install dependencies¶
Uncomment and run the following cell to install the required dependencies.
# %pip install segment-geospatial
import leafmap
from samgeo import SamGeo2, raster_to_vector, regularize
Create an interactive map¶
m = leafmap.Map(center=[47.653287, -117.588070], zoom=16, height="800px")
m.add_basemap("Satellite")
m
Download a sample image¶
Pan and zoom the map to select the area of interest. Use the draw tools to draw a polygon or rectangle on the map
if m.user_roi is not None:
bbox = m.user_roi_bounds()
else:
bbox = [-117.6029, 47.65, -117.5936, 47.6563]
image = "satellite.tif"
leafmap.map_tiles_to_geotiff(
output=image, bbox=bbox, zoom=18, source="Satellite", overwrite=True
)
You can also use your own image. Uncomment and run the following cell to use your own image.
# image = '/path/to/your/own/image.tif'
Display the downloaded image on the map.
m.layers[-1].visible = False
m.add_raster(image, layer_name="Image")
m
Initialize SAM class¶
Set automatic=False to enable the SAM2ImagePredictor.
sam = SamGeo2(
model_id="sam2-hiera-large",
automatic=False,
)
Specify the image to segment.
sam.set_image(image)
Display the map. Use the drawing tools to draw some rectangles around the features you want to extract, such as trees, buildings.
m
Create bounding boxes¶
If no rectangles are drawn, the default bounding boxes will be used as follows:
if m.user_rois is not None:
boxes = m.user_rois
else:
boxes = [
[-117.5995, 47.6518, -117.5988, 47.652],
[-117.5987, 47.6518, -117.5979, 47.652],
]
Segment the image¶
Use the predict() method to segment the image with specified bounding boxes. The boxes parameter accepts a list of bounding box coordinates in the format of [[left, bottom, right, top], [left, bottom, right, top], ...], a GeoJSON dictionary, or a file path to a GeoJSON file.
sam.predict(boxes=boxes, point_crs="EPSG:4326", output="mask.tif", dtype="uint8")
Display the result¶
Add the segmented image to the map.
m.add_raster("mask.tif", cmap="viridis", nodata=0, layer_name="Mask")
m
Use an existing vector file as box prompts¶
Alternatively, you can specify a file path to a vector file. Let's download a sample vector file from GitHub.
url = "https://github.com/opengeos/datasets/releases/download/samgeo/building_bboxes.geojson"
geojson = "building_bboxes.geojson"
leafmap.download_file(url, geojson)
Display the vector data on the map.
m = leafmap.Map()
m.add_raster(image, layer_name="Image")
style = {
"color": "#ffff00",
"weight": 2,
"fillColor": "#7c4185",
"fillOpacity": 0,
}
m.add_vector(geojson, style=style, zoom_to_layer=True, layer_name="Bboxes")
m
Segment image with box prompts¶
Segment the image using the specified file path to the vector mask.
output_masks = "building_masks.tif"
sam.predict(
boxes=geojson,
point_crs="EPSG:4326",
output=output_masks,
dtype="uint8",
multimask_output=False,
)
Display the segmented masks on the map.
m.add_raster(
output_masks, cmap="jet", nodata=0, opacity=0.5, layer_name="Building masks"
)
m
Convert raster to vector¶
output_vector = "building_vector.geojson"
raster_to_vector(output_masks, output_vector)
Regularize building footprints¶
output_regularized = "building_regularized.geojson"
regularize(output_vector, output_regularized)
m.add_vector(
output_regularized, style=style, layer_name="Building regularized", info_mode=None
)