{ "cells": [ { "cell_type": "markdown", "id": "a66708a3", "metadata": {}, "source": [ "# Searching the STAC Catalog\n", "\n", "Authors: Aimee Barciauskas (Development Seed)\n", "\n", "Date: December 13, 2022\n", "\n", "Description: This tutorial provides a basic introduction to searching the [MAAP STAC catalog](https://stac.maap-project.org/) using `pystac-client`.\n", "\n", "Another method of searching the STAC catalog is via the [STAC browser](https://stac-browser.maap-project.org).\n", "\n", "\"Drawing\"\n" ] }, { "cell_type": "markdown", "id": "521aa28d", "metadata": {}, "source": [ "## About the STAC Catalog\n", "\n", "The MAAP STAC catalog provides discovery of a subset of MAAP datasets. These collections are hosted specifically through the MAAP STAC catalog and are typically not available on NASA's CMR. The data files have not been moved at all in the process of publishing datasets to STAC.\n", "\n", "Data will continue to be added to the STAC catalog with priority given to datasets which are known to be in-use by MAAP UWG members through S3 metrics, direct collaboration with data team members, and by request." ] }, { "cell_type": "markdown", "id": "789cb9a5", "metadata": {}, "source": [ "## Additional Resources\n", "- [Pystac-client Introduction](https://pystac-client.readthedocs.io/en/stable/tutorials/pystac-client-introduction.html)" ] }, { "cell_type": "markdown", "id": "27b6af81", "metadata": {}, "source": [ "## Importing and Installing Packages\n", "\n", "In order to run this notebook you'll need the following packages:" ] }, { "cell_type": "code", "execution_count": 1, "id": "e16c7c99", "metadata": {}, "outputs": [], "source": [ "%%capture\n", "%pip install -U pystac-client" ] }, { "cell_type": "code", "execution_count": 2, "id": "d20f803e", "metadata": {}, "outputs": [], "source": [ "from pystac_client import Client" ] }, { "cell_type": "markdown", "id": "51a18e54", "metadata": {}, "source": [ "## STAC Client\n", "We first connect to an API by retrieving the root catalog, or landing page, of the API with the Client.open function." ] }, { "cell_type": "code", "execution_count": 3, "id": "404a8ab6", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", " \n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# STAC API root URL\n", "URL = 'https://stac.maap-project.org/'\n", "cat = Client.open(URL)\n", "cat" ] }, { "cell_type": "markdown", "id": "f34b7a57", "metadata": {}, "source": [ "### Searching Collections\n", "\n", "As with a static catalog the get_collections function will iterate through the Collections in the Catalog. Notice that because this is an API it can get all the Collections through a single call, rather than having to fetch each one individually." ] }, { "cell_type": "code", "execution_count": 4, "id": "9f0e2ae9", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "stac_collections = list(cat.get_collections())\n", "stac_collections" ] }, { "cell_type": "code", "execution_count": 5, "id": "6fcdcd1a", "metadata": {}, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "
\n", "
\n", "
    \n", " \n", " \n", " \n", "
  • \n", " type\n", " \"Collection\"\n", "
    • \n", " \n", " \n", " \n", " \n", " \n", "
  • \n", " id\n", " \"Landsat8_SurfaceReflectance\"\n", "
    • \n", " \n", " \n", " \n", " \n", " \n", "
  • \n", " stac_version\n", " \"1.0.0\"\n", "
    • \n", " \n", " \n", " \n", " \n", " \n", "
  • \n", " description\n", " \"Landsat Analysis Ready Data (ARD) are consistently processed to the highest scientific standards and level of processing required for direct use in monitoring and assessing landscape change. A fundamental goal for Landsat ARD is to significantly reduce the magnitude of data processing for application scientists, who currently have to download and prepare large amounts of Landsat scene-based data for time-series investigative analysis.This Landsat ARD dataset consists of Landsat 8 Operational Land Imager (OLI) Surface Reflectance are generated using the Landsat Surface Reflectance Code (LaSRC) code. Surface reflectance (SR) is generated for the AfriSAR region of interest and consistently processed using per pixel solar zenith angle corrections and gridded to a common cartographic projection.This collection is a subset over Peru (N: -4.73978, S: -9.72818, E: -75.58157, W: -79.80472) and equatorial western Africa (N: 3.93979, S: -5.39294, E: 16.50647, W: 7.9348) for 12 April 2013 to 29 January 2020.\"\n", "
    • \n", " \n", " \n", " \n", " \n", "
  • \n", " \n", " links\n", " [] 4 items\n", " \n", " \n", "
      \n", " \n", " \n", " \n", "
    • \n", " 0\n", "
        \n", " \n", " \n", " \n", "
      • \n", " rel\n", " \"items\"\n", "
        • \n", " \n", " \n", " \n", " \n", " \n", "
      • \n", " href\n", " \"https://stac.maap-project.org/collections/Landsat8_SurfaceReflectance/items\"\n", "
        • \n", " \n", " \n", " \n", " \n", " \n", "
      • \n", " type\n", " \"application/geo+json\"\n", "
        • \n", " \n", " \n", " \n", "
      \n", "
      • \n", " \n", " \n", " \n", "
    \n", " \n", "
      \n", " \n", " \n", " \n", "
    • \n", " 1\n", "
        \n", " \n", " \n", " \n", "
      • \n", " rel\n", " \"parent\"\n", "
        • \n", " \n", " \n", " \n", " \n", " \n", "
      • \n", " href\n", " \"https://stac.maap-project.org/\"\n", "
        • \n", " \n", " \n", " \n", " \n", " \n", "
      • \n", " type\n", " \"application/json\"\n", "
        • \n", " \n", " \n", " \n", "
      \n", "
      • \n", " \n", " \n", " \n", "
    \n", " \n", "
      \n", " \n", " \n", " \n", "
    • \n", " 2\n", "
        \n", " \n", " \n", " \n", "
      • \n", " rel\n", " \"root\"\n", "
        • \n", " \n", " \n", " \n", " \n", " \n", "
      • \n", " href\n", " \"https://stac.maap-project.org/\"\n", "
        • \n", " \n", " \n", " \n", " \n", " \n", "
      • \n", " type\n", " \"application/json\"\n", "
        • \n", " \n", " \n", " \n", " \n", " \n", "
      • \n", " title\n", " \"MAAP STAC API (dev)\"\n", "
        • \n", " \n", " \n", " \n", "
      \n", "
      • \n", " \n", " \n", " \n", "
    \n", " \n", "
      \n", " \n", " \n", " \n", "
    • \n", " 3\n", "
        \n", " \n", " \n", " \n", "
      • \n", " rel\n", " \"self\"\n", "
        • \n", " \n", " \n", " \n", " \n", " \n", "
      • \n", " href\n", " \"https://stac.maap-project.org/collections/Landsat8_SurfaceReflectance\"\n", "
        • \n", " \n", " \n", " \n", " \n", " \n", "
      • \n", " type\n", " \"application/json\"\n", "
        • \n", " \n", " \n", " \n", "
      \n", "
      • \n", " \n", " \n", " \n", "
    \n", " \n", "
    • \n", " \n", " \n", " \n", " \n", "
  • \n", " item_assets\n", "
      \n", " \n", "
    \n", "
    • \n", " \n", " \n", " \n", " \n", " \n", "
  • \n", " title\n", " \"Landsat 8 Operational Land Imager (OLI) Surface Reflectance Analysis Ready Data (ARD) V1, Peru and Equatorial Western Africa, April 2013-January 2020\"\n", "
    • \n", " \n", " \n", " \n", " \n", " \n", "
  • \n", " extent\n", "
      \n", " \n", " \n", " \n", "
    • \n", " spatial\n", "
        \n", " \n", " \n", "
      • \n", " \n", " bbox\n", " [] 1 items\n", " \n", " \n", "
          \n", " \n", " \n", "
        • \n", " \n", " 0\n", " [] 4 items\n", " \n", " \n", "
            \n", " \n", " \n", " \n", "
          • \n", " 0\n", " -79.80472\n", "
            • \n", " \n", " \n", " \n", "
          \n", " \n", "
            \n", " \n", " \n", " \n", "
          • \n", " 1\n", " -9.72818\n", "
            • \n", " \n", " \n", " \n", "
          \n", " \n", "
            \n", " \n", " \n", " \n", "
          • \n", " 2\n", " 16.50647\n", "
            • \n", " \n", " \n", " \n", "
          \n", " \n", "
            \n", " \n", " \n", " \n", "
          • \n", " 3\n", " 3.93979\n", "
            • \n", " \n", " \n", " \n", "
          \n", " \n", "
          • \n", " \n", " \n", "
        \n", " \n", "
        • \n", " \n", " \n", "
      \n", "
      • \n", " \n", " \n", " \n", " \n", " \n", "
    • \n", " temporal\n", "
        \n", " \n", " \n", "
      • \n", " \n", " interval\n", " [] 1 items\n", " \n", " \n", "
          \n", " \n", " \n", "
        • \n", " \n", " 0\n", " [] 2 items\n", " \n", " \n", "
            \n", " \n", " \n", " \n", "
          • \n", " 0\n", " \"2013-04-12T09:28:35.404000Z\"\n", "
            • \n", " \n", " \n", " \n", "
          \n", " \n", "
            \n", " \n", " \n", " \n", "
          • \n", " 1\n", " \"2020-01-29T00:00:00Z\"\n", "
            • \n", " \n", " \n", " \n", "
          \n", " \n", "
          • \n", " \n", " \n", "
        \n", " \n", "
        • \n", " \n", " \n", "
      \n", "
      • \n", " \n", " \n", " \n", "
    \n", "
    • \n", " \n", " \n", " \n", " \n", " \n", "
  • \n", " license\n", " \"not-provided\"\n", "
    • \n", " \n", " \n", " \n", "
\n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "collection = cat.get_collection(stac_collections[0].id)\n", "collection" ] }, { "cell_type": "markdown", "id": "3f4e956f", "metadata": {}, "source": [ "### Searching STAC Items\n", "\n", "Query the `/search` endpoint of the STAC catalog to find items in our collection. This method will return an ItemSearch instance which we can then turn into a list.\n", "\n", "Read more about additional parameters to the `search()` method at [pystac-client.readthedocs.io](https://pystac-client.readthedocs.io/en/stable/api.html#pystac_client.Client.search)." ] }, { "cell_type": "code", "execution_count": 6, "id": "a939e555", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ,\n", " ]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "collection_items = list(cat.search(collections=[collection.id], max_items=10).items())\n", "collection_items" ] }, { "cell_type": "markdown", "id": "98234222", "metadata": {}, "source": [ "We can get a specific item by supplying one of the IDs from an item in our previous collection search. We are then able to get the HREF of the first asset in our item.\n" ] }, { "cell_type": "code", "execution_count": 7, "id": "a636fdd6", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'s3://nasa-maap-data-store/file-staging/nasa-map/Landsat8_SurfaceReflectance___1/LC080090662019122401T1-SC20200127151508.tar.gz'" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "item = collection.get_item(collection_items[0].id)\n", "item.assets[list(item.assets.keys())[0]].href" ] }, { "cell_type": "markdown", "id": "0a223d73", "metadata": {}, "source": [ "Here's a simplified example:" ] }, { "cell_type": "code", "execution_count": null, "id": "9b6553db", "metadata": {}, "outputs": [], "source": [ "# Retrieve a specific collection\n", "collection = cat.get_collection(\"ESACCI_Biomass_L4_AGB_V4_100m\")\n", "\n", "# Search for items in the collection\n", "collection_items = list(cat.search(collections=[\"ESACCI_Biomass_L4_AGB_V4_100m\"], max_items=10).items())\n", "\n", "# Retrieve a specific item\n", "item = collection.get_item(\"S50W080_ESACCI-BIOMASS-L4-AGB-MERGED-100m-2020-fv4.0\")\n", "\n", "# List the item's asset href\n", "item.assets[\"estimates\"].href" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3.9.16 ('maap-docs': conda)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" }, "vscode": { "interpreter": { "hash": "9c680c3978eba20fff90e7e5c9e7d02604367989d0ed0a42fa5791bddc7acaa7" } } }, "nbformat": 4, "nbformat_minor": 5 }