{ "cells": [ { "cell_type": "markdown", "id": "71971df7", "metadata": {}, "source": [ "# ATLAS/ICESat-02 ATL08 Access and Visualize\n", "\n", "Author: Sumant Jha (MSFC/USRA), Alex Mandel (Development Seed), Jamison French (Development Seed), Rajat Shinde (UAH), Sheyenne Kirkland (UAH)\n", "\n", "Date: March 7, 2024\n", "\n", "Description: In this example, we'll walk through accessing ATL08 data using both cloud access and through downloading. We'll then explore the data using `H5py`, `h5Glance`, and `xarray`. Finally, we'll visualize some of the data using `xarray`." ] }, { "cell_type": "markdown", "id": "1d840225-f6db-4077-bfa3-94b6921e7265", "metadata": {}, "source": [ "## Run This Notebook\n", "\n", "To access and run this tutorial within MAAP's Algorithm Development Environment (ADE), please refer to the [\"Getting started with the MAAP\"](https://docs.maap-project.org/en/latest/getting_started/getting_started.html) section of our documentation.\n", "\n", "Disclaimer: This tutorial will use an experimental feature to allow access to the DAAC without using EarthDataLogin. This tutorial will need to be ran within MAAP's ADE to allow this experimental feature to work. Running the tutorial outside of the MAAP ADE will result in errors. Additionally, it is recommended to use the `Pangeo` workspace within the MAAP ADE." ] }, { "cell_type": "markdown", "id": "ff8efa52-24ef-42ce-b616-4d259039569e", "metadata": {}, "source": [ "## About the Data\n", "\n", "This data set (ATL08) contains along-track heights above the WGS84 ellipsoid (ITRF2014 reference frame) for the ground and canopy surfaces. The canopy and ground surfaces are processed in fixed 100 m data segments, which typically contain more than 100 signal photons. The data were acquired by the Advanced Topographic Laser Altimeter System (ATLAS) instrument on board the Ice, Cloud and land Elevation Satellite-2 (ICESat-2) observatory.\n", "\n", "```\n", "Parameter(s): TERRAIN ELEVATION\n", "Platform(s):ICESat-2\n", "Sensor(s): ATLAS\n", "Data Format(s): HDF5\n", "Temporal Coverage: 14 October 2018 to present\n", "Temporal Resolution: 91 day\n", "Spatial Resolution: Varies\n", "Spatial Reference System(s): WGS 84 EPSG:4326\n", "Spatial Coverage: N: 90 S: -90 E: 180 W: -180\n", "```\n", "\n", "(Source: [ATL08 v6 Dataset Landing Page](https://nsidc.org/data/atl08/versions/5))\n" ] }, { "cell_type": "markdown", "id": "24b1f8cd-53f1-43f6-b85d-1224576e8605", "metadata": {}, "source": [ "## Additional Resources\n", "- [Earthdata Search](https://search.earthdata.nasa.gov/search?q=atl08&ff=Available%20in%20Earthdata%20Cloud)\n", "- [ATL08 v6 User Guide](https://nsidc.org/sites/default/files/documents/user-guide/atl08-v006-userguide.pdf)" ] }, { "cell_type": "markdown", "id": "d0c3d7fb-6d7f-4027-9c73-15379dfa9d62", "metadata": {}, "source": [ "## Importing and Installing Packages" ] }, { "cell_type": "markdown", "id": "bceb37a2", "metadata": {}, "source": [ "The following example uses several packages. If you do not have the following packages, uncomment the code below:" ] }, { "cell_type": "code", "execution_count": null, "id": "3a9b3e53-c67b-4294-999a-04e7cc2c149c", "metadata": { "tags": [] }, "outputs": [], "source": [ "# !pip install -q h5py h5glance requests fsspec s3fs h5netcdf" ] }, { "cell_type": "markdown", "id": "69ee9aed-39a7-42fa-aa85-e5185a7ffc7f", "metadata": {}, "source": [ "Optional download (do not need to download if you're using the Pangeo workspace. Uncomment if using other workspaces):" ] }, { "cell_type": "code", "execution_count": null, "id": "c9665604-ce0e-47b1-b7b9-5648d62a6b2e", "metadata": {}, "outputs": [], "source": [ "# !pip install -q rioxarray" ] }, { "cell_type": "code", "execution_count": 11, "id": "43dc9655", "metadata": {}, "outputs": [], "source": [ "import os\n", "import h5py\n", "from maap.maap import MAAP\n", "from h5glance import H5Glance\n", "import requests\n", "import xarray\n", "import boto3\n", "import fsspec" ] }, { "cell_type": "markdown", "id": "9a01c263-4b64-4eb5-8844-672456f3fec0", "metadata": {}, "source": [ "## Accessing the Data" ] }, { "cell_type": "markdown", "id": "ab76b7e7-408e-45c3-b506-d193cb505aab", "metadata": {}, "source": [ "In this section we'll walk through two different ways to access the data. The first method is to access the data through S3.\n", "\n", "The second method is to access it by using `maap-py`, then download the data locally. From there, we will explore some of the data using `h5glance`." ] }, { "cell_type": "markdown", "id": "ff2227d4", "metadata": {}, "source": [ "### Example 1: Accessing the Data with S3\n", "\n", "We will use role assumption to gain access to the data. To do this, we'll use an experimental feature by setting up a parameter, assuming a role, and get temporary credentials with the assumed role. We'll also set up credentials into fsspec so we can later use `xarray` for data exploration." ] }, { "cell_type": "code", "execution_count": 12, "id": "23c1abf0-0369-4833-85ef-3e556409959a", "metadata": {}, "outputs": [], "source": [ "def assume_role_credentials(ssm_parameter_name):\n", " # Create a session using your current credentials\n", " session = boto3.Session()\n", "\n", " # Retrieve the SSM parameter\n", " ssm = session.client('ssm', \"us-west-2\")\n", " parameter = ssm.get_parameter(\n", " Name=ssm_parameter_name, \n", " WithDecryption=True\n", " )\n", " parameter_value = parameter['Parameter']['Value']\n", "\n", " # Assume the DAAC access role\n", " sts = session.client('sts')\n", " assumed_role_object = sts.assume_role(\n", " RoleArn=parameter_value,\n", " RoleSessionName='TutorialSession'\n", " )\n", "\n", " # From the response that contains the assumed role, get the temporary \n", " # credentials that can be used to make subsequent API calls\n", " credentials = assumed_role_object['Credentials']\n", "\n", " return credentials\n", "\n", "def fsspec_access(credentials):\n", " # Pass assumed role credentials into fsspec\n", " return fsspec.filesystem(\n", " \"s3\",\n", " key=credentials['AccessKeyId'],\n", " secret=credentials['SecretAccessKey'],\n", " token=credentials['SessionToken']\n", " )" ] }, { "cell_type": "code", "execution_count": 13, "id": "9d93b99d-37a3-4f9e-a4e8-4939a8275506", "metadata": { "tags": [] }, "outputs": [], "source": [ "s3_fsspec = fsspec_access(assume_role_credentials(\"/iam/maap-data-reader\"))" ] }, { "cell_type": "markdown", "id": "6d45ac1b", "metadata": {}, "source": [ "### Example 2: Download Data Locally" ] }, { "cell_type": "markdown", "id": "3ffb06f4", "metadata": {}, "source": [ "We are going to use NASA host which is NASA's Common Metadata Repository (CMR) to search for and download ICESat data. \n", "ICESat's ATL08 data's concept id can be found on https://search.earthdata.nasa.gov/search and looking for 'ATL08' in the search bar. When you check the metadata associated with your search result, you can get the concept_id associated with below tutorial. In this case, the concept_id is `C2613553260-NSIDC_CPRD`. \n", "\n", "For this example, we are going to use granule id of `ATL08_20230816182927_08792008_006_01.h5`. This will be in HDF5 format. \n", "\n", "With all this information in hand, we are ready to make a query to cmr.earthdata.nasa.gov using `maap-py`. \n" ] }, { "cell_type": "code", "execution_count": 14, "id": "e87a6048", "metadata": {}, "outputs": [], "source": [ "maap = MAAP(maap_host='api.maap-project.org')\n", "\n", "nasa_host = \"cmr.earthdata.nasa.gov\"\n", "results = maap.searchGranule(cmr_host=nasa_host,\n", " concept_id=\"C2613553260-NSIDC_CPRD\",\n", " readable_granule_name=\"ATL08_20230816182927_08792008_006_01.h5\")" ] }, { "cell_type": "markdown", "id": "1b5b0235", "metadata": {}, "source": [ "Let's see how this turned out. Did we get a result?" ] }, { "cell_type": "code", "execution_count": 15, "id": "1fff1d07", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'concept-id': 'G2800397933-NSIDC_CPRD',\n", " 'collection-concept-id': 'C2613553260-NSIDC_CPRD',\n", " 'revision-id': '2',\n", " 'format': 'application/echo10+xml',\n", " 'Granule': {'GranuleUR': 'ATL08_20230816182927_08792008_006_01.h5',\n", " 'InsertTime': '2023-11-11T10:14:36.642Z',\n", " 'LastUpdate': '2023-11-11T10:14:36.642Z',\n", " 'Collection': {'DataSetId': 'ATLAS/ICESat-2 L3A Land and Vegetation Height V006'},\n", " 'DataGranule': {'SizeMBDataGranule': '106.29328060150146',\n", " 'ProducerGranuleId': 'ATL08_20230816182927_08792008_006_01.h5',\n", " 'DayNightFlag': 'UNSPECIFIED',\n", " 'ProductionDateTime': '2023-09-21T19:57:30.000Z'},\n", " 'Temporal': {'RangeDateTime': {'BeginningDateTime': '2023-08-16T18:29:24.813Z',\n", " 'EndingDateTime': '2023-08-16T18:36:27.524Z'}},\n", " 'Spatial': {'HorizontalSpatialDomain': {'Geometry': {'GPolygon': {'Boundary': {'Point': [{'PointLongitude': '-49.71155',\n", " 'PointLatitude': '-0.55384'},\n", " {'PointLongitude': '-51.21715', 'PointLatitude': '-15.49643'},\n", " {'PointLongitude': '-52.36359', 'PointLatitude': '-26.40442'},\n", " {'PointLongitude': '-52.43242', 'PointLatitude': '-27.04278'},\n", " {'PointLongitude': '-52.5585', 'PointLatitude': '-27.03176'},\n", " {'PointLongitude': '-52.48902', 'PointLatitude': '-26.39347'},\n", " {'PointLongitude': '-51.3338', 'PointLatitude': '-15.48562'},\n", " {'PointLongitude': '-49.82395', 'PointLatitude': '-0.54264'},\n", " {'PointLongitude': '-49.76474', 'PointLatitude': '0.04913'},\n", " {'PointLongitude': '-49.65236', 'PointLatitude': '0.03773'}]}}}}},\n", " 'OnlineAccessURLs': {'OnlineAccessURL': [{'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-protected/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01.h5',\n", " 'MimeType': 'application/x-hdf5'},\n", " {'URL': 's3://nsidc-cumulus-prod-protected/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01.h5',\n", " 'MimeType': 'application/x-hdf5'}]},\n", " 'OnlineResources': {'OnlineResource': [{'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-protected/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01.h5.dmrpp',\n", " 'Type': 'USER SUPPORT',\n", " 'MimeType': 'application/vnd.opendap.dap4.dmrpp+xml'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01.iso.xml',\n", " 'Type': 'USER SUPPORT',\n", " 'MimeType': 'text/xml'},\n", " {'URL': 's3://nsidc-cumulus-prod-protected/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01.h5.dmrpp',\n", " 'Type': 'USER SUPPORT',\n", " 'MimeType': 'application/vnd.opendap.dap4.dmrpp+xml'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01.iso.xml',\n", " 'Type': 'USER SUPPORT',\n", " 'MimeType': 'text/xml'},\n", " {'URL': 'https://opendap.earthdata.nasa.gov/collections/C2613553260-NSIDC_CPRD/granules/ATL08_20230816182927_08792008_006_01.h5',\n", " 'Type': 'USER SUPPORT'}]},\n", " 'AssociatedBrowseImageUrls': {'ProviderBrowseUrl': [{'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.default.default1.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.default.default2.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1l.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1l.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1l.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1l.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1l.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1r.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1r.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1r.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1r.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1r.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2l.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2l.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2l.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2l.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2l.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2r.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2r.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2r.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2r.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2r.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3l.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3l.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3l.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3l.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3l.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3r.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3r.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3r.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3r.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 'https://data.nsidc.earthdatacloud.nasa.gov/nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3r.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.default.default1.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.default.default2.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1l.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1l.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1l.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1l.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1l.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1r.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1r.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1r.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1r.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt1r.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2l.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2l.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2l.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2l.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2l.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2r.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2r.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2r.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2r.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt2r.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3l.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3l.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3l.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3l.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3l.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3r.groundtrack.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3r.h_canopy_abs.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3r.h_te_median.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3r.n_ca_photons.jpg',\n", " 'MimeType': 'image/jpeg'},\n", " {'URL': 's3://nsidc-cumulus-prod-public/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01_BRW.gt3r.n_te_photons.jpg',\n", " 'MimeType': 'image/jpeg'}]}}}" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "data_file = results[0]\n", "data_file" ] }, { "cell_type": "markdown", "id": "163283a5", "metadata": {}, "source": [ "Looks like we did get a result and will be able to learn a lot about it from available metadata. Let's download the HDF file locally.\n", "\n", "Establish a temporary directory to store the data file and display the path and filename:" ] }, { "cell_type": "code", "execution_count": 16, "id": "419c6e78", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'./data/ATL08_20230816182927_08792008_006_01.h5'" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "dataDir = './data'\n", "if not os.path.exists(dataDir): os.mkdir(dataDir)\n", "data = data_file.getData(dataDir)\n", "data" ] }, { "cell_type": "markdown", "id": "188a7017-ed15-44f4-8d25-9bfe5ca43524", "metadata": {}, "source": [ "## Exploring the Data\n", "\n", "There are two different ways we'll open and look at the data:\n", "1. Using `xarray`\n", "2. Using `h5py`" ] }, { "cell_type": "markdown", "id": "f424754e-5b2e-4b82-ba94-dead8e39b887", "metadata": {}, "source": [ "### 1. xarray" ] }, { "cell_type": "markdown", "id": "fd45311f-64ce-4afc-a664-396e33bdf56f", "metadata": {}, "source": [ "First, let's grab the S3 URL for direct access from our results above." ] }, { "cell_type": "code", "execution_count": 17, "id": "162ed41f", "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'s3://nsidc-cumulus-prod-protected/ATLAS/ATL08/006/2023/08/16/ATL08_20230816182927_08792008_006_01.h5'" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "s3_url = results[0]['Granule']['OnlineAccessURLs']['OnlineAccessURL'][1]['URL']\n", "s3_url" ] }, { "cell_type": "markdown", "id": "6e2308ef-97c6-40a1-9cc8-2ddf77f5538d", "metadata": {}, "source": [ "Now we can open a specific group within the HDF5 file using `xarray`." ] }, { "cell_type": "code", "execution_count": 18, "id": "6d30ef03-4878-4e9b-beef-b535476db455", "metadata": { "tags": [] }, "outputs": [ { "data": { "text/html": [ "
<xarray.Dataset>\n",
"Dimensions: (delta_time: 18690, ds_geosegments: 5, ds_surf_type: 5)\n",
"Coordinates:\n",
" * delta_time (delta_time) datetime64[ns] 2023-08-16T18:29:24.853128...\n",
" latitude (delta_time) float32 ...\n",
" longitude (delta_time) float32 ...\n",
"Dimensions without coordinates: ds_geosegments, ds_surf_type\n",
"Data variables: (12/41)\n",
" asr (delta_time) float32 ...\n",
" atlas_pa (delta_time) float32 ...\n",
" beam_azimuth (delta_time) float32 ...\n",
" beam_coelev (delta_time) float32 ...\n",
" brightness_flag (delta_time) float32 ...\n",
" cloud_flag_atm (delta_time) float32 ...\n",
" ... ...\n",
" snr (delta_time) float32 ...\n",
" solar_azimuth (delta_time) float32 ...\n",
" solar_elevation (delta_time) float32 ...\n",
" surf_type (delta_time, ds_surf_type) int8 ...\n",
" terrain_flg (delta_time) float64 ...\n",
" urban_flag (delta_time) float64 ...\n",
"Attributes:\n",
" Description: Contains data categorized as land at 100 meter intervals.\n",
" data_rate: Data are stored as aggregates of 100 meters.