AfriSAR Search and Visualize

Authors: Nikita Susan (UAH), Aimee Barciauskas (DevSeed)

Date: January 17, 2023

Description: In this tutorial, we will search for AfriSAR AGB (Above Ground Biomass) data and download a TIFF file from the ORNL DAAC S3. The TIFF file will then be read in with rioxarray and visualized using hvplot.

Run This Notebook

To access and run this tutorial within MAAP’s Algorithm Development Environment (ADE), please refer to the “Getting started with the MAAP” section of our documentation.

Disclaimer: it is highly recommended to run a tutorial within MAAP’s ADE, which already includes packages specific to MAAP, such as maap-py. Running the tutorial outside of the MAAP ADE may lead to errors.

About the Data

“This dataset provides gridded estimates of aboveground biomass (AGB) for four sites in Gabon at 0.25 ha (50 m) resolution derived with field measurements and airborne LiDAR data collected from 2010 to 2016. The sites represent a mix of forested, savannah, and some agricultural and disturbed landcover types: Lope site, within Lope National Park; Mabounie, mostly forested site; Mondah Forest, protected area; and the Rabi forest site, part of the Smithsonian Institution of Global Earth Observatories world-wide network of forest plots. Plot-level biophysical measurements of tree diameter and tree height (or estimated by allometry) were performed at 1 ha and 0.25 ha scales on multiple plots at each site and used to derive AGB for each tree and then summed for each plot. Aerial LiDAR scans were used to construct digital elevation models (DEM) and digital surface models (DSM), and then the DEM and DSM were used to construct a canopy height model (CHM) at 1 m resolution. After checking site-plot locations against the CHM, mean canopy height (MCH) was computed over each 0.25 ha. A single regression model relating MCH and AGB estimates, incorporating local height based on the trunk DBH (HD) relationships, was produced for all sites and combined with the CHM layer to construct biomass maps at 0.25 ha resolution.” (Source: AfriSAR AGB User Guide)

Additional Resources

Import and Install Packages

First, let’s import and install packages. If you don’t have the packages below installed already, uncomment the following line.

[ ]:
# !pip install rioxarray hvplot
[38]:
import rioxarray
import rasterio as rio
import hvplot.xarray
from maap.maap import MAAP
import boto3
from rasterio.session import AWSSession
import os

import warnings
warnings.filterwarnings("ignore")

Search for AfriSAR AGB Data

Using MAAP’s searchCollection function and the collection short name, we’ll pull in the AfriSAR_AGB_Maps_1681 collection.

[8]:
maap = MAAP(maap_host='api.maap-project.org')
[83]:
results = maap.searchCollection(cmr_host='cmr.earthdata.nasa.gov', short_name='AfriSAR_AGB_Maps_1681')
results
[83]:
[{'concept-id': 'C2734261660-ORNL_CLOUD',
  'revision-id': '2',
  'format': 'application/echo10+xml',
  'Collection': {'ShortName': 'AfriSAR_AGB_Maps_1681',
   'VersionId': '1',
   'InsertTime': '2022-11-28T00:00:00Z',
   'LastUpdate': '2023-07-17T18:24:39Z',
   'LongName': 'AfriSAR: Aboveground Biomass for Lope, Mabounie, Mondah, and Rabi Sites, Gabon',
   'DataSetId': 'AfriSAR: Aboveground Biomass for Lope, Mabounie, Mondah, and Rabi Sites, Gabon',
   'Description': 'This dataset provides gridded estimates of aboveground biomass (AGB) for four sites in Gabon at 0.25 ha (50 m) resolution derived with field measurements and airborne LiDAR data collected from 2010 to 2016. The sites represent a mix of forested, savannah, and some agricultural and disturbed landcover types: Lope site, within Lope National Park; Mabounie, mostly forested site; Mondah Forest, protected area; and the Rabi forest site, part of the Smithsonian Institution of Global Earth Observatories world-wide network of forest plots. Plot-level biophysical measurements of tree diameter and tree height (or estimated by allometry) were performed at 1 ha and 0.25 ha scales on multiple plots at each site and used to derive AGB for each tree and then summed for each plot. Aerial LiDAR scans were used to construct digital elevation models (DEM) and digital surface models (DSM), and then the DEM and DSM were used to construct a canopy height model (CHM) at 1 m resolution. After checking site-plot locations against the CHM, mean canopy height (MCH) was computed over each 0.25 ha. A single regression model relating MCH and AGB estimates, incorporating local height based on the trunk DBH (HD) relationships, was produced for all sites and combined with the CHM layer to construct biomass maps at 0.25 ha resolution.',
   'DOI': {'DOI': '10.3334/ORNLDAAC/1681', 'Authority': 'https://doi.org'},
   'Orderable': 'false',
   'Visible': 'true',
   'RevisionDate': '2022-11-28T00:00:00Z',
   'ProcessingLevelId': '3',
   'ProcessingLevelDescription': 'Variables mapped on uniform space-time grid scales with completeness and consistency',
   'ArchiveCenter': 'ORNL_DAAC',
   'CitationForExternalPublication': 'Saatchi, S.S., J. Chave, N. Labriere, N. Barbier, M. Maxime-Rejou, A. Ferraz, and S. Tao. 2019. AfriSAR: Aboveground Biomass for Lope, Mabounie, Mondah, and Rabi Sites, Gabon. ORNL DAAC, Oak Ridge, Tennessee, USA. https://doi.org/10.3334/ORNLDAAC/1681',
   'CollectionState': 'COMPLETE',
   'MaintenanceAndUpdateFrequency': 'As needed',
   'UseConstraints': {'LicenseURL': {'URL': 'https://science.nasa.gov/earth-science/earth-science-data/data-information-policy',
     'Description': 'License URL for data use policy',
     'Type': 'Data Use Policy',
     'MimeType': 'text/html'}},
   'Price': '0',
   'DataFormat': 'GeoTIFF',
   'SpatialKeywords': {'Keyword': 'GABON'},
   'Temporal': {'RangeDateTime': {'BeginningDateTime': '2016-02-01T00:00:00Z',
     'EndingDateTime': '2016-03-31T23:59:59Z'}},
   'Contacts': {'Contact': {'Role': 'ARCHIVER',
     'OrganizationName': 'ORNL_DAAC',
     'OrganizationAddresses': {'Address': {'StreetAddress': 'ORNL DAAC User Services Office, P.O. Box 2008, MS 6407, Oak Ridge National Laboratory',
       'City': 'Oak Ridge',
       'StateProvince': 'Tennessee',
       'PostalCode': '37831-6407',
       'Country': 'USA'}},
     'OrganizationPhones': {'Phone': {'Number': '(865) 241-3952',
       'Type': 'Direct Line'}},
     'OrganizationEmails': {'Email': 'uso@daac.ornl.gov'}}},
   'ScienceKeywords': {'ScienceKeyword': [{'CategoryKeyword': 'EARTH SCIENCE',
      'TopicKeyword': 'BIOSPHERE',
      'TermKeyword': 'ECOSYSTEMS',
      'VariableLevel1Keyword': {'Value': 'TERRESTRIAL ECOSYSTEMS',
       'VariableLevel2Keyword': {'Value': 'FORESTS'}}},
     {'CategoryKeyword': 'EARTH SCIENCE',
      'TopicKeyword': 'BIOSPHERE',
      'TermKeyword': 'VEGETATION',
      'VariableLevel1Keyword': {'Value': 'BIOMASS'}},
     {'CategoryKeyword': 'EARTH SCIENCE',
      'TopicKeyword': 'BIOSPHERE',
      'TermKeyword': 'VEGETATION',
      'VariableLevel1Keyword': {'Value': 'CARBON'}}]},
   'Platforms': {'Platform': [{'ShortName': 'FIELD SURVEYS',
      'LongName': 'FIELD SURVEYS',
      'Type': 'Field Sites',
      'Instruments': {'Instrument': {'ShortName': 'STEEL MEASURING TAPE',
        'LongName': 'STEEL MEASURING TAPE'}}},
     {'ShortName': 'Airplane',
      'LongName': 'Airplane',
      'Type': 'Jet',
      'Instruments': {'Instrument': {'ShortName': 'LIDAR',
        'LongName': 'Light Detection and Ranging'}}},
     {'ShortName': 'B-200',
      'LongName': 'Beechcraft King Air B-200',
      'Type': 'Propeller',
      'Instruments': {'Instrument': {'ShortName': 'LVIS',
        'LongName': 'Land, Vegetation, and Ice Sensor'}}}]},
   'Campaigns': {'Campaign': {'ShortName': 'AfriSAR', 'LongName': 'AfriSAR'}},
   'OnlineAccessURLs': {'OnlineAccessURL': {'URL': 'https://daac.ornl.gov/afrisar/AfriSAR_AGB_Maps/',
     'URLDescription': 'This link allows direct data access via Earthdata login'}},
   'OnlineResources': {'OnlineResource': [{'URL': 'https://daac.ornl.gov/AFRISAR/guides/AfriSAR_AGB_Maps.html',
      'Description': 'ORNL DAAC Data Set Documentation',
      'Type': "USER'S GUIDE"},
     {'URL': 'https://doi.org/10.3334/ORNLDAAC/1681',
      'Description': 'Data set Landing Page DOI URL',
      'Type': 'DATA SET LANDING PAGE'},
     {'URL': 'https://data.ornldaac.earthdata.nasa.gov/public/afrisar/AfriSAR_AGB_Maps/comp/AfriSAR_AGB_Maps.pdf',
      'Description': 'AfriSAR: Aboveground Biomass Maps for Lope, Mabounie, Mondah, and Rabi, Gabon, 2016: AfriSAR_AGB_Maps.pdf',
      'Type': 'GENERAL DOCUMENTATION'},
     {'URL': 'https://data.ornldaac.earthdata.nasa.gov/public/afrisar/AfriSAR_AGB_Maps/comp/AfriSAR_AGB_Maps_PlotDetails.csv',
      'Description': 'AfriSAR: Aboveground Biomass for Lope, Mabounie, Mondah, and Rabi Sites, Gabon: AfriSAR_AGB_Maps_PlotDetails.csv',
      'Type': 'GENERAL DOCUMENTATION'},
     {'URL': 'https://daac.ornl.gov/AFRISAR/guides/AfriSAR_AGB_Maps_Fig1.png',
      'Description': 'Aboveground biomass (AGB) map at 0.25-ha resolution for the Mondah study site. Source: Mabounie_AGB_50m.tif',
      'Type': 'GET RELATED VISUALIZATION',
      'MimeType': 'image/png'}]},
   'Spatial': {'SpatialCoverageType': 'HORIZONTAL',
    'HorizontalSpatialDomain': {'Geometry': {'CoordinateSystem': 'CARTESIAN',
      'BoundingRectangle': {'WestBoundingCoordinate': '9.30',
       'NorthBoundingCoordinate': '0.61',
       'EastBoundingCoordinate': '11.64',
       'SouthBoundingCoordinate': '-1.95'}}},
    'GranuleSpatialRepresentation': 'CARTESIAN'},
   'DirectDistributionInformation': {'Region': 'us-west-2',
    'S3BucketAndObjectPrefixName': 's3://ornl-cumulus-prod-protected/afrisar/AfriSAR_AGB_Maps/',
    'S3CredentialsAPIEndpoint': 'https://data.ornldaac.earthdata.nasa.gov/s3credentials',
    'S3CredentialsAPIDocumentationURL': 'https://data.ornldaac.earthdata.nasa.gov/s3credentialsREADME'}}}]

Using the searchGranule function and the concept-id from our collection search, we can also discover granules within the collection. For this tutorial, we’ll be visualizing the third granule in the collection, so let’s retrieve that one. This granule is of the Rabi forest site.

[84]:
granules = maap.searchGranule(cmr_host = 'cmr.earthdata.nasa.gov', concept_id = 'C2734261660-ORNL_CLOUD')
granules[3]
[84]:
{'concept-id': 'G2734344223-ORNL_CLOUD',
 'collection-concept-id': 'C2734261660-ORNL_CLOUD',
 'revision-id': '1',
 'format': 'application/echo10+xml',
 'Granule': {'GranuleUR': 'AfriSAR_AGB_Maps.Rabi_AGB_50m.tif',
  'InsertTime': '2022-11-28T00:00:00Z',
  'LastUpdate': '2023-07-17T18:24:45Z',
  'Collection': {'ShortName': 'AfriSAR_AGB_Maps_1681', 'VersionId': '1'},
  'DataGranule': {'DataGranuleSizeInBytes': '14109',
   'SizeMBDataGranule': '0.014109',
   'Checksum': {'Value': '514dca209ed19076e5bdf2595af86af2a76d7a318ad76cc56480fc4a8bb26fba',
    'Algorithm': 'SHA-256'},
   'DayNightFlag': 'BOTH',
   'ProductionDateTime': '2022-11-28T00:00:00Z'},
  'Temporal': {'RangeDateTime': {'BeginningDateTime': '2016-02-01T00:00:00Z',
    'EndingDateTime': '2016-03-31T23:59:59Z'}},
  'Spatial': {'HorizontalSpatialDomain': {'Geometry': {'BoundingRectangle': {'WestBoundingCoordinate': '9.85914',
      'NorthBoundingCoordinate': '-1.90031',
      'EastBoundingCoordinate': '9.90636',
      'SouthBoundingCoordinate': '-1.94602'}}}},
  'MeasuredParameters': {'MeasuredParameter': [{'ParameterName': 'FORESTS'},
    {'ParameterName': 'CARBON'},
    {'ParameterName': 'BIOMASS'}]},
  'Platforms': {'Platform': [{'ShortName': 'FIELD SURVEYS',
     'Instruments': {'Instrument': {'ShortName': 'STEEL MEASURING TAPE'}}},
    {'ShortName': 'Airplane',
     'Instruments': {'Instrument': {'ShortName': 'LIDAR'}}},
    {'ShortName': 'B-200',
     'Instruments': {'Instrument': {'ShortName': 'LVIS'}}}]},
  'Campaigns': {'Campaign': {'ShortName': 'AfriSAR'}},
  'Price': '0',
  'OnlineAccessURLs': {'OnlineAccessURL': [{'URL': 'https://data.ornldaac.earthdata.nasa.gov/protected/afrisar/AfriSAR_AGB_Maps/data/Rabi_AGB_50m.tif',
     'URLDescription': 'Download Rabi_AGB_50m.tif',
     'MimeType': 'image/tiff'},
    {'URL': 'https://data.ornldaac.earthdata.nasa.gov/public/afrisar/AfriSAR_AGB_Maps/data/Rabi_AGB_50m.tif.sha256',
     'URLDescription': 'Download Rabi_AGB_50m.tif.sha256'},
    {'URL': 's3://ornl-cumulus-prod-protected/afrisar/AfriSAR_AGB_Maps/data/Rabi_AGB_50m.tif',
     'URLDescription': 'This link provides direct download access via S3 to the granule',
     'MimeType': 'image/tiff'}]},
  'OnlineResources': {'OnlineResource': [{'URL': 'https://daac.ornl.gov/AFRISAR/guides/AfriSAR_AGB_Maps.html',
     'Description': 'ORNL DAAC Data Set Documentation',
     'Type': "USER'S GUIDE"},
    {'URL': 'https://doi.org/10.3334/ORNLDAAC/1681',
     'Description': 'Data set Landing Page DOI URL',
     'Type': 'DATA SET LANDING PAGE'},
    {'URL': 'https://daac.ornl.gov/daacdata/afrisar/AfriSAR_AGB_Maps/comp/AfriSAR_AGB_Maps.pdf',
     'Description': 'Data Set Documentation',
     'Type': 'GENERAL DOCUMENTATION'},
    {'URL': 'https://daac.ornl.gov/daacdata/afrisar/AfriSAR_AGB_Maps/comp/AfriSAR_AGB_Maps_PlotDetails.csv',
     'Description': 'Data Set Documentation',
     'Type': 'GENERAL DOCUMENTATION'},
    {'URL': 'https://data.ornldaac.earthdata.nasa.gov/s3credentials',
     'Description': 'api endpoint to retrieve temporary credentials valid for same-region direct s3 access',
     'Type': 'VIEW RELATED INFORMATION'}]},
  'Orderable': 'false',
  'DataFormat': 'COG'}}

Download the Granule File

We’ll download our file directly from the ORNL DAAC S3.

Let’s pull in the collection and file name for the granule of interest.

[70]:
granule_ur=granules[3]['Granule']['GranuleUR'].split(".")
collection_name=granule_ur[0]
file_name=granule_ur[1]
[71]:
print(f"collection name: {collection_name} | file_name: {file_name}")
collection name: AfriSAR_AGB_Maps | file_name: Rabi_AGB_50m

Next, we will request temporary s3 credentials for the ORNL DAAC. Once these credentials are given, the file can be downloaded to our workspace.

[72]:
def get_s3_creds(url):
    return maap.aws.earthdata_s3_credentials(url)

def get_s3_client(s3_cred_endpoint):
    creds=get_s3_creds(s3_cred_endpoint)
    boto3_session = boto3.Session(
            aws_access_key_id=creds['accessKeyId'],
            aws_secret_access_key=creds['secretAccessKey'],
            aws_session_token=creds['sessionToken']
    )
    return boto3_session.client("s3")

def download_s3_file(s3, bucket, collection_name, file_name):
    os.makedirs("/projects/afrisar", exist_ok=True) # create directories, as necessary
    download_path=f"/projects/afrisar/{file_name}.tif"
    s3.download_file(bucket, f"afrisar/{collection_name}/data/{file_name}.tif", download_path)
    return download_path
[73]:
s3_cred_endpoint= 'https://data.ornldaac.earthdata.nasa.gov/s3credentials'
s3=get_s3_client(s3_cred_endpoint)
[74]:
bucket="ornl-cumulus-prod-protected"
download_path=download_s3_file(s3, bucket, collection_name, file_name)
download_path
[74]:
'/projects/afrisar/Rabi_AGB_50m.tif'

Read and Visualize

Read in our file with rioxarray…

[75]:
da = rioxarray.open_rasterio(download_path)
da = da.squeeze('band', drop=True)
da
[75]:
<xarray.DataArray (y: 101, x: 105)>
[10605 values with dtype=float32]
Coordinates:
  * x            (x) float64 5.956e+05 5.956e+05 ... 6.007e+05 6.008e+05
  * y            (y) float64 9.79e+06 9.79e+06 9.79e+06 ... 9.785e+06 9.785e+06
    spatial_ref  int64 0
Attributes:
    AREA_OR_POINT:  Area
    _FillValue:     -9999.0
    scale_factor:   1.0
    add_offset:     0.0

… and visualize our data using hvplot.

[76]:
ds_masked = da.where(da != da._FillValue)

ds_masked.hvplot(
    'x', 'y',
    cmap='viridis',
    frame_height=400,
    frame_width=400
).redim.range(value=(0,da.max().values))
[76]: