{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3bb79280",
   "metadata": {},
   "source": [
    "# Accessing Data Provided by NASA's Distributed Active Archive Centers (DAACs)\n",
    "\n",
    "Authors: Samuel Ayers (UAH), Alex Mandel (Development Seed)\n",
    "\n",
    "Date: April 3, 2023\n",
    "\n",
    "Description: In this example, we will demonstrate how to get authorized access from a Distributed Active Archive Center (DAAC) in NASA Earthdata, and then access data from the DAAC. Note: not all DAACs will require prior authorization to access data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b39baf6-b797-435e-b970-aafe89f52dbc",
   "metadata": {},
   "source": [
    "## Run This Notebook\n",
    "To access and run this tutorial within MAAP's Algorithm Development Environment (ADE), please refer to the [\"Getting started with the MAAP\"](https://docs.maap-project.org/en/latest/getting_started/getting_started.html) section of our documentation.\n",
    "\n",
    "Disclaimer: it is highly recommended to run a tutorial within MAAP's ADE, which already includes packages specific to MAAP, such as maap-py. Running the tutorial outside of the MAAP ADE may lead to errors."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efbf2e09-d514-44e8-aa62-00aeaebd8187",
   "metadata": {},
   "source": [
    "## Additional Resources\n",
    "- [Changes to Land Processes Distributed Active Archive Center (LP DAAC) Data Access](https://lpdaac.usgs.gov/news/important-changes-to-lp-daac-data-access/)\n",
    "- [LP DAAC AppEEARS API Documentation](https://appeears.earthdatacloud.nasa.gov/api/?_gl=1*1nu2qiw*_ga*NDQ0Nzk1MDgzLjE2OTMyMzg3NjE.*_ga_0YWDZEJ295*MTY5MzIzODc2MS4xLjEuMTY5MzIzOTMyOS4wLjAuMA..)\n",
    "- [Alaska Satellite Facility (ASF) API Basics](https://docs.asf.alaska.edu/api/basics/)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42790f93-b0ae-4c32-aa4e-5af61540d403",
   "metadata": {},
   "source": [
    "## Create a Workspace"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "857d06d6",
   "metadata": {},
   "source": [
    "It is possible to download data provided by [DAACs](https://www.earthdata.nasa.gov/eosdis/daacs), including data which is not cataloged by the MAAP's CMR, using the [NASA MAAP ADE](https://ade.maap-project.org/). This data is hosted externally from the MAAP but can be accessed using the NASA MAAP ADE's authentication systems.\n",
    "\n",
    "In order to do this, we start by creating a Jupyter workspace within the NASA MAAP ADE. Using the left-hand navigation, select \"+ Get Started\" and then select the \"Jupyter - MAAP Basic Stable\" workspace.\n",
    "\n",
    "![Create Jupyter Workspace](../../_static/create_jupyter_workspace.png)\n",
    "\n",
    "Alternatively, you can create a workspace using the \"Workspaces\" interface. See [Create Workspace](https://docs.maap-project.org/en/latest/system_reference_guide/create_workspace.html) for more information."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca9e80e0",
   "metadata": {},
   "source": [
    "## Importing Packages"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22c00f9b",
   "metadata": {},
   "source": [
    "You can access data from Jupyter Notebook within your workspace. In your Jupyter Notebook, start by importing the `maap` package. Then invoke the `MAAP` constructor, setting the `maap_host` argument to `'api.maap-project.org'`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "47ee4a37",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import the maap package\n",
    "from maap.maap import MAAP\n",
    "#import os for directory creation later in the example\n",
    "import os\n",
    "# invoke the MAAP constructor using the maap_host argument\n",
    "maap = MAAP(maap_host='api.maap-project.org')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d9958a7-f390-45ae-9ca2-cb67a9a4e629",
   "metadata": {},
   "source": [
    "## Using maap-py outside MAAP ADE"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1baa1450-e822-4cb8-a2fe-6d9540154822",
   "metadata": {},
   "source": [
    "To use `maap-py` outside the MAAP ADE, you need to pass the MAAP host explicitly and set the `MAAP_PGT` token in your environment. \n",
    "\n",
    "The token can be obtained on the [MAAP profile](https://maap-project.org/profile/). Then, before running any code that accesses protected data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b608d94-8c59-42a3-b7ec-b68ec3342b2e",
   "metadata": {},
   "outputs": [],
   "source": [
    "export MAAP_PGT=\"your_PGT_token\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "554308e6-2df0-4838-8f80-8541ba85e90c",
   "metadata": {},
   "source": [
    "This token will be used by `maap.download_granule()` internally to fetch data from NASA-hosted S3 buckets. While this is done automatically in MAAP ADE, outside users must ensure the token is correctly exported into the environment.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7b5f903",
   "metadata": {},
   "source": [
    "## Granting Earthdata Login Access"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c508114",
   "metadata": {},
   "source": [
    "In order to access external DAAC data from the NASA MAAP ADE, MAAP uses your Earthdata Login profile to send a data request to the desired DAAC application. \n",
    "\n",
    "Some DAAC applications (such as 'Alaska Satellite Facility Data Access') must be authorized before you can use them. Login or register at https://urs.earthdata.nasa.gov/ in order to see the applications that you have authorized. From the profile page, click on the 'Applications' tab and select 'Authorized Apps' from the drop-down menu.\n",
    "\n",
    "![profile page](../../_static/profile_page.png)\n",
    "\n",
    "This takes you to the Approved Applications page which lists the applications you have authorized. To add more applications, scroll down to the bottom of the page and click the 'APPROVE MORE APPLICATIONS' button which takes you to the Application search page. \n",
    "\n",
    "![Approved Applications page](../../_static/approved_applications_page.png)\n",
    "\n",
    "Enter the desired application name within the search box and click the 'SEARCH' button. After this, a list of search results appears.\n",
    "\n",
    "![Application search page](../../_static/application_search_page.png)\n",
    "\n",
    "Once you find the desired application, click the 'AUTHORIZE' button next to the name. \n",
    "\n",
    "![search results](../../_static/search_results.png)\n",
    "\n",
    "You are then presented with its End User License Agreement. In order to have authorization, you need to select the 'I agree to the terms of End User License Agreement' checkbox and then click the 'AGREE' button.\n",
    "\n",
    "![End User License Agreement Page](../../_static/end_user_license_agreement_page.png)\n",
    "\n",
    "Along with 'Alaska Satellite Facility Data Access', you must also authorize 'Alaska Satellite Facility Data Access Egress Control'. Return to the search page to find this application.\n",
    "\n",
    "![egress_control_asf](../../_static/egress_control_asf.png)\n",
    "\n",
    "Similar to the previous application, once you click 'AUTHORIZE', you will also select the 'I agree to the terms of the End User License Agreement' checkbox and click 'AGREE'.\n",
    "\n",
    "After these steps are done, you are then shown the Approved Applications page again and both of the desired applications should now be listed.\n",
    "\n",
    "![approved_asf_apps](../../_static/approved_asf_apps.png)\n",
    "\n",
    "Note that if Earthdata Login access is not granted to your target DAAC application, the following example will result in a 401-permission error."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "72b2b987",
   "metadata": {},
   "source": [
    "## Accessing Sentinel-1 Granule Data from the Alaska Satellite Facility (ASF)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef0390b0",
   "metadata": {},
   "source": [
    "Search for a granule using the `searchGranule` function (for more information on searching for granules, see [Searching for Granules in MAAP](https://docs.maap-project.org/en/latest/technical_tutorials/search/granules.html)). Then utilize the `getData` function, which downloads granule data if it doesn't already exist locally. We can use `getData` to download the first result from our granule search into the file system and assign it to a variable (in this case `download`). Note that you will need to authorize the 'Alaska Satellite Facility Data Access' application before downloading any results from our search (see the above section for more information concerning authorizing applications).\n",
    "\n",
    "Before downloading our data, we will also create a new data directory to download our files into using `os`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "f7c0d770-bad5-4f74-af73-8998f71a43b8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# set data directory\n",
    "dataDir = './data'\n",
    "\n",
    "# check if directory exists -> if directory doesn't exist, directory is created\n",
    "if not os.path.exists(dataDir):\n",
    "    os.mkdir(dataDir)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "6378e8de",
   "metadata": {},
   "outputs": [],
   "source": [
    "# search for granule data using the short_name argument \n",
    "results = maap.searchGranule(short_name='SENTINEL-1A_DP_GRD_HIGH')\n",
    "# download first result\n",
    "download = results[0].getData(dataDir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b871373f",
   "metadata": {},
   "source": [
    "Note that we can then use the `print` function to see the file name and directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "e831ede2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./data/S1A_S3_GRDH_1SDH_20140615T034444_20140615T034512_001055_00107C_8977.zip\n"
     ]
    }
   ],
   "source": [
    "# print file directory\n",
    "print(download)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "abfe4af6",
   "metadata": {},
   "source": [
    "## Accessing Harmonized Landsat Sentinel-2 (HLS) Level 3 Granule Data from the Land Processes Distributed Active Archive Center (LP DAAC)\n",
    "\n",
    "We use a similar approach in order to access HLS Level 3 granule data. Note that this data is not cataloged by the MAAP's CMR but we can use `searchGranule`'s `cmr_host` argument to specify a CMR instance external to MAAP."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "924c0b46",
   "metadata": {},
   "outputs": [],
   "source": [
    "# search for granule data using CMR host name and short name arguments\n",
    "results = maap.searchGranule(\n",
    "    cmr_host='cmr.earthdata.nasa.gov',\n",
    "    short_name='HLSL30')\n",
    "# download first result\n",
    "download = results[0].getData(dataDir)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef64180f",
   "metadata": {},
   "source": [
    "As in the previous example, we can use the `print` function to see the file name and directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "e08ce66e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./data/HLS.L30.T59WPT.2013101T001445.v2.0.B09.tif\n"
     ]
    }
   ],
   "source": [
    "# print file directory\n",
    "print(download)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}