{ "cells": [ { "cell_type": "markdown", "id": "fa96e18d-c80f-4b67-9595-eeab44cf079f", "metadata": {}, "source": [ "# Accessing the MAAP CMR STAC with R\n", "\n", "Authors: Harshini Girish (UAH), Sheyenne Kirkland (UAH), Alex Mandel (Development Seed), Henry Rodman (Development Seed)\n", "\n", "Date: December 13, 2024\n", "\n", "Description: In this notebook, we'll use `rstac` to search for collections and associated items within the [MAAP STAC Catalog](https://stac.maap-project.org/)." ] }, { "cell_type": "markdown", "id": "bed79c96-e264-4c4c-b77e-5ca4693fcac1", "metadata": {}, "source": [ "## Run This Notebook\n", "\n", "To access and run this tutorial within MAAP's Algorithm Development Environment (ADE), please refer to the [\"Getting started with the MAAP\"](https://docs.maap-project.org/en/latest/getting_started/getting_started.html) section of our documentation.\n", "\n", "Disclaimer: it is highly recommended to run a tutorial within MAAP's ADE, which already includes packages specific to MAAP, such as maap-py. Running the tutorial outside of the MAAP ADE may lead to errors. Users should work within the \"R/Python\" workspace." ] }, { "cell_type": "markdown", "id": "8120c211-1570-47a1-8b63-ad4fd6559709", "metadata": {}, "source": [ "## Additional Resources\n", "\n", "- [How do I find data using R?](https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/find-data/find-r.html)\n", " - A resource from NASA Openscapes, showing users how to search for NASA data in R and get authentication using the package `earthdatalogin`. Additionally, it shows users how to find data stored in NASA STACs (SpatioTemporal Asset Catalogs).\n", "- [rstac: Client Library for SpatioTemporal Asset Catalog](https://cran.r-project.org/web/packages/rstac/index.html)\n", " - A page with materials for the `rstac` library.\n", "- [Searching the STAC Catalog (MAAP Docs)](https://docs.maap-project.org/en/latest/technical_tutorials/search/searching_the_stac_catalog.html)\n", " - A notebook in the MAAP Docs that shows users how to search the MAAP STAC using Python." ] }, { "cell_type": "markdown", "id": "5f681c27-f7fd-42cb-974a-9bd107136c99", "metadata": {}, "source": [ " ## Install/Load Packages\n", "\n", "Let's install and load the packages necessary for this tutorial." ] }, { "cell_type": "code", "execution_count": null, "id": "a8980e56-7012-4ed6-8601-6b4c2824c52c", "metadata": {}, "outputs": [], "source": [ "install.packages(\"rstac\")\n", "\n", "library(rstac)" ] }, { "cell_type": "markdown", "id": "6ec6e591-a0b2-4b1e-b449-e59825f21065", "metadata": {}, "source": [ "## Initializing the MAAP STAC Endpoint\n", "Before beginning, we'll form a connection to the MAAP STAC endpoint to set up and inspect the STAC endpoint for querying geospatial data." ] }, { "cell_type": "code", "execution_count": 224, "id": "6befa70a-6afb-4e88-aee8-ff274c68b9f5", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "STAC Endpoint Metadata:\n", "\u001b[1m###rstac_query\u001b[22m\n", "- \u001b[1murl:\u001b[22m https://stac.maap-project.org/\n", "- \u001b[1mparams:\u001b[22m\n", "- \u001b[1mfield(s):\u001b[22m version, base_url, endpoint, params, verb, encode\n" ] } ], "source": [ "# Define the MAAP STAC endpoint\n", "stac_endpoint <- stac(\"https://stac.maap-project.org/\")\n", "\n", "# Display the STAC endpoint metadata\n", "cat(\"STAC Endpoint Metadata:\\n\")\n", "print(stac_endpoint)" ] }, { "cell_type": "markdown", "id": "a69f2346-7834-4088-9070-ce3573ee99f0", "metadata": {}, "source": [ "## Fetching and Displaying STAC Collections\n", "\n", "This code fetches and displays collections from a STAC (SpatioTemporal Asset Catalog) endpoint. It extracts id and title for each collection for further exploration or querying.\n", "\n" ] }, { "cell_type": "code", "execution_count": 225, "id": "8efb7817-42ba-4d16-9b62-53be30f13637", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collection ID: Landsat8_SurfaceReflectance \n", "Title: Landsat 8 Operational Land Imager (OLI) Surface Reflectance Analysis Ready Data (ARD) V1, Peru and Equatorial Western Africa, April 2013-January 2020 \n", "\n", "Collection ID: Global_PALSAR2_PALSAR_FNF \n", "Title: Global 25m Resolution PALSAR-2/PALSAR Forest/Non-Forest Map \n", "\n", "Collection ID: Global_Forest_Change_2000-2017 \n", "Title: Global Forest Change 2000-2017 \n", "\n", "Collection ID: AFRISAR_DLR2 \n", "Title: AFRISAR_DLR2 \n", "\n", "Collection ID: GlobCover_09 \n", "Title: GlobCover Global Land Cover Product (2009) \n", "\n", "Collection ID: AfriSAR_UAVSAR_KZ \n", "Title: AfriSAR UAVSAR Vertical Wavenumber (KZ) Generated Using NISAR Tools \n", "\n", "Collection ID: AfriSAR_UAVSAR_Ungeocoded_Covariance \n", "Title: AfriSAR UAVSAR Ungeocoded Covariance Matrix product Generated Using NISAR Tools \n", "\n", "Collection ID: AfriSAR_UAVSAR_Normalization_Area \n", "Title: AfriSAR UAVSAR Normalization Area Generated Using NISAR Tools \n", "\n", "Collection ID: AfriSAR_UAVSAR_Geocoded_SLC \n", "Title: AfriSAR UAVSAR Geocoded SLCs Generated Using NISAR Tools \n", "\n", "Collection ID: AfriSAR_UAVSAR_Geocoded_Covariance \n", "Title: AfriSAR UAVSAR Geocoded Covariance Matrix product Generated Using NISAR Tools \n", "\n", "Collection ID: GlobCover_05_06 \n", "Title: GlobCover Global Land Cover Product (2005-06) \n", "\n", "Collection ID: GEDI_CalVal_Field_Data \n", "Title: Global Ecosystem Dynamics Investigation (GEDI) Calibration/Validation Field Survey Dataset \n", "\n", "Collection ID: AfriSAR_UAVSAR_Coreg_SLC \n", "Title: AfriSAR UAVSAR Coregistered SLCs Generated Using NISAR Tools \n", "\n", "Collection ID: GEDI_CalVal_Lidar_Data_Compressed \n", "Title: Global Ecosystem Dynamics Investigation (GEDI) Calibration/Validation Airborne Lidar Dataset (Compressed) \n", "\n", "Collection ID: ABoVE_UAVSAR_PALSAR \n", "Title: Arctic-Boreal Vulnerability Experiment Uninhabited Aerial Vehicle Synthetic Aperture Radar Polarimetric SAR \n", "\n", "Collection ID: AFRISAR_DLR \n", "Title: AFRISAR_DLR \n", "\n", "Collection ID: BIOSAR1 \n", "Title: BIOSAR1 \n", "\n", "Collection ID: GEDI_CalVal_Lidar_Data \n", "Title: Global Ecosystem Dynamics Investigation (GEDI) Calibration/Validation Airborne Lidar Dataset \n", "\n", "Collection ID: ICESat2_Boreal_AGB_tindex_average \n", "Title: ICESat2-Boreal Above Ground Biomass T-Index Average \n", "\n", "Collection ID: Paraguay_Country_Pilot \n", "Title: Paraguay Country Pilot \n", "\n", "Collection ID: ESACCI_Biomass_L4_AGB_V4_100m \n", "Title: ESA CCI Above-Ground Biomass Product Level 4 Version 4 \n", "\n", "Collection ID: NCEO_Africa_AGB_100m_2017 \n", "Title: National Centre for Eath Oberservation (NCEO) Africa Aboveground Woody Biomass (AGB) 2017 \n", "\n", "Collection ID: SRTMGL1_COD \n", "Title: NASA Shuttle Radar Topography Mission Global 1 \n", "\n", "Collection ID: GEDI_CalVal_Lidar_COPC \n", "Title: GEDI CalVal Lidar COPC \n", "\n", "Collection ID: icesat2-boreal \n", "Title: Gridded Boreal Aboveground Biomass Density c.2020 at 30m resolution \n", "\n", "Collection ID: nisar-sim \n", "Title: Simulated NISAR \n", "\n", "Collection ID: glad-glclu2020-v2 \n", "Title: GLAD: Annual maps of land cover and land use \n", "\n", "Collection ID: glad-glclu2020-change-v2 \n", "Title: GLAD: Net change of land cover and land use between 2000 and 2020 \n", "\n" ] } ], "source": [ "collections <- stac_endpoint |>\n", " collections() |>\n", " get_request()\n", "# Ensure collections are retrieved\n", "if (length(collections$collections) > 0) {\n", " # Extract collection IDs and titles\n", " collection_info <- lapply(collections$collections, function(x) {\n", " list(id = x$id, title = x$title)\n", " })\n", " # Display the collection information\n", " for (i in seq_along(collection_info)) {\n", " cat(\"Collection ID:\", collection_info[[i]]$id, \"\\n\")\n", " cat(\"Title:\", collection_info[[i]]$title, \"\\n\\n\")\n", " }\n", "} else {\n", " cat(\"No collections found or error retrieving collections.\\n\")\n", "}" ] }, { "cell_type": "markdown", "id": "9f3249c0-c48f-492a-810f-30b19ddbd825", "metadata": {}, "source": [ "## Assigning and Selecting a STAC Collection ID\n", "This code selects a collection ID from the list of collections retrieved from the STAC catalog. It selects a single collection ID from the fetched collections.\n", "\n" ] }, { "cell_type": "code", "execution_count": 226, "id": "e2ac9c3e-9009-49ca-98b2-18c6c85c2b65", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Selected Collection ID: ESACCI_Biomass_L4_AGB_V4_100m \n" ] } ], "source": [ "# Assign collection ID\n", "if (length(collections$collections) > 0) {\n", " # Choose a specific collection (21st in this example)\n", " collection_id <- collections$collections[[21]]$id\n", " cat(\"Selected Collection ID:\", collection_id, \"\\n\")\n", "} else {\n", " stop(\"No collections found.\")\n", "}" ] }, { "cell_type": "markdown", "id": "6dee1538-b808-4a7b-be1f-45df34cd6b50", "metadata": {}, "source": [ "## Searching and Retrieving Items from a STAC Collection\n", "This code searches for items in the selected STAC collection using the stac_search() function. It safely handles errors during the query and retrieves the items, printing details such as item IDs, dates, and associated links. If no items are found, it outputs a message indicating so." ] }, { "cell_type": "code", "execution_count": 227, "id": "f35b7e4b-1fe4-4a50-bba4-54b98930f3cd", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m###Items\u001b[22m\n", "- \u001b[1mfeatures\u001b[22m (10 item(s)):\n", " - S50W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", " - S50W070_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", " - S50W060_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", " - S50W040_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", " - S50E070_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", " - S50E060_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", " - S40W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", " - S40W070_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", " - S40E170_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", " - S40E160_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", "- \u001b[1massets:\u001b[22m estimates, standard_deviation\n", "- \u001b[1mitem's fields:\u001b[22m \n", "assets, bbox, collection, geometry, id, links, properties, stac_extensions, stac_version, type\n", "Found 10 items:\n", "\n", "Item ID: S50W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 \n", "Date: 2020-01-01T00:00:00+00:00 \n", "Links: NULL, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m, https://stac.maap-project.org/, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m/items/S50W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 \n", "\n", "Item ID: S50W070_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 \n", "Date: 2020-01-01T00:00:00+00:00 \n", "Links: NULL, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m, https://stac.maap-project.org/, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m/items/S50W070_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 \n", "\n", "Item ID: S50W060_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 \n", "Date: 2020-01-01T00:00:00+00:00 \n", "Links: NULL, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m, https://stac.maap-project.org/, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m/items/S50W060_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 \n", "\n", "Item ID: S50W040_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 \n", "Date: 2020-01-01T00:00:00+00:00 \n", "Links: NULL, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m, https://stac.maap-project.org/, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m/items/S50W040_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 \n", "\n", "Item ID: S50E070_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 \n", "Date: 2020-01-01T00:00:00+00:00 \n", "Links: NULL, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m, https://stac.maap-project.org/, https://stac.maap-project.org/collections/ESACCI_Biomass_L4_AGB_V4_100m/items/S50E070_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 \n", "\n" ] } ], "source": [ "# Fetch items\n", "items <- stac_endpoint |>\n", " stac_search(collections = collection_id) |>\n", " get_request()\n", "# Print the retrieved items\n", "print(items)\n", "# Process and display item information\n", "if (length(items$features) > 0) {\n", " cat(\"Found\", length(items$features), \"items:\\n\\n\")\n", " # Display details of the first few items\n", " for (i in seq_len(min(5, length(items$features)))) {\n", " item <- items$features[[i]]\n", " cat(\"Item ID:\", item$id, \"\\n\")\n", " cat(\"Date:\", item$properties$datetime, \"\\n\")\n", " cat(\"Links:\", paste(sapply(item$links, function(x) x$href), collapse = \", \"), \"\\n\\n\")\n", " }\n", "} else {\n", " cat(\"No items found for collection:\", collection_id, \"\\n\")\n", "}" ] }, { "cell_type": "markdown", "id": "27dbea15-2b4b-4966-b708-fa87802e82df", "metadata": {}, "source": [ "## Extracting and Displaying Assets from a STAC Item\n", "This code extracts the assets (downloadable data resources) from the first item in the STAC search results." ] }, { "cell_type": "code", "execution_count": 228, "id": "be0bacd4-3b0c-4b7e-a5b9-4d6695d44acb", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[1m###Item\u001b[22m\n", "- \u001b[1mid:\u001b[22m S50W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\n", "- \u001b[1mcollection:\u001b[22m ESACCI_Biomass_L4_AGB_V4_100m\n", "- \u001b[1mbbox:\u001b[22m \n", "xmin: -80.00000, ymin: -60.00000, xmax: -70.00000, ymax: -50.00000\n", "- \u001b[1mdatetime:\u001b[22m 2020-01-01T00:00:00+00:00\n", "- \u001b[1massets:\u001b[22m estimates, standard_deviation\n", "- \u001b[1mitem's fields:\u001b[22m \n", "assets, bbox, collection, geometry, id, links, properties, stac_extensions, stac_version, type\n", "Available assets:\n", "[1] \"estimates\" \"standard_deviation\"\n" ] } ], "source": [ "if (length(items$features) > 0) {\n", " # Extract and display the first item's assets\n", " first_item <- items$features[[1]]\n", " assets <- first_item$assets\n", " print(first_item)\n", " cat(\"Available assets:\\n\")\n", " print(names(assets)) # List of asset types\n", "} else {\n", " cat(\"No items found for collection:\", collection_id, \"\\n\")\n", "}" ] }, { "cell_type": "markdown", "id": "becf66fc-4057-475c-baed-df46528f0ec3", "metadata": {}, "source": [ "## Listing and Displaying Asset URLs from a STAC Item\n", "This loop iterates through all available assets in the STAC item and prints each asset's name and its corresponding URL." ] }, { "cell_type": "code", "execution_count": 229, "id": "ec8e8369-0d96-4079-99e7-8421eb180913", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Asset: estimates \n", "URL: s3://nasa-maap-data-store/file-staging/nasa-map/ESACCI_Biomass_L4_AGB_V4_100m_2020/S50W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0.tif \n", "\n", "Asset: standard_deviation \n", "URL: s3://nasa-maap-data-store/file-staging/nasa-map/ESACCI_Biomass_L4_AGB_V4_100m_2020/S50W080_ESACCI-BIOMASS-L4-AGB_SD-MERGED-100m-2020-fv4.0.tif \n", "\n" ] } ], "source": [ "for (asset_name in names(assets)) {\n", " cat(\"Asset:\", asset_name, \"\\n\")\n", " cat(\"URL:\", assets[[asset_name]]$href, \"\\n\\n\")\n", "}" ] }, { "cell_type": "markdown", "id": "02046460-ebd0-4589-bf76-4b842a05c99c", "metadata": {}, "source": [ "## Performing a Focused Search Using the MAAP STAC API\n", "\n", "This code performs a search query and retrieves items from the MAAP STAC. The search is configured with the following parameters:\n", "\n", "Collection: Specifies the dataset to search within.\n", "\n", "Temporal Range: Filters items within a specific date range.\n", "\n", "Bounding Box: Spatially filters items to a defined area of interest." ] }, { "cell_type": "code", "execution_count": 230, "id": "faa5c6a6-d613-42c3-bfe5-8a8d4e538dce", "metadata": {}, "outputs": [ { "data": { "text/html": [ "
| collection | id | datetime | desc |
|---|---|---|---|
| <chr> | <chr> | <chr> | <chr> |
| ESACCI_Biomass_L4_AGB_V4_100m | S50W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 | 2020-01-01T00:00:00+00:00 | Cloud Optimized GeoTIFF of AGB estimates |
| ESACCI_Biomass_L4_AGB_V4_100m | S50W070_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 | 2020-01-01T00:00:00+00:00 | Cloud Optimized GeoTIFF of AGB estimates |
| ESACCI_Biomass_L4_AGB_V4_100m | S50W060_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 | 2020-01-01T00:00:00+00:00 | Cloud Optimized GeoTIFF of AGB estimates |
| ESACCI_Biomass_L4_AGB_V4_100m | S50W040_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 | 2020-01-01T00:00:00+00:00 | Cloud Optimized GeoTIFF of AGB estimates |
| ESACCI_Biomass_L4_AGB_V4_100m | S40W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 | 2020-01-01T00:00:00+00:00 | Cloud Optimized GeoTIFF of AGB estimates |
| ESACCI_Biomass_L4_AGB_V4_100m | S40W070_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 | 2020-01-01T00:00:00+00:00 | Cloud Optimized GeoTIFF of AGB estimates |
| ESACCI_Biomass_L4_AGB_V4_100m | S30W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 | 2020-01-01T00:00:00+00:00 | Cloud Optimized GeoTIFF of AGB estimates |
| ESACCI_Biomass_L4_AGB_V4_100m | S30W070_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 | 2020-01-01T00:00:00+00:00 | Cloud Optimized GeoTIFF of AGB estimates |
| ESACCI_Biomass_L4_AGB_V4_100m | S30W060_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 | 2020-01-01T00:00:00+00:00 | Cloud Optimized GeoTIFF of AGB estimates |
| ESACCI_Biomass_L4_AGB_V4_100m | S20W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0 | 2020-01-01T00:00:00+00:00 | Cloud Optimized GeoTIFF of AGB estimates |