{ "cells": [ { "cell_type": "markdown", "id": "cf2fa006-7f2d-407f-8490-2435b36fa55c", "metadata": {}, "source": [ "# Jobs UI for Job Management\n", "\n", "The Jobs UI allows users to submit and view DPS jobs from their Jupyter workspace. Users can monitor job status, access generated products, view errors, and view other job metadata." ] }, { "cell_type": "markdown", "id": "db3a214d-4c06-426e-a00c-1ffc8fec73b2", "metadata": {}, "source": [ "## Access the Jobs UI\n", "\n", "1. From your workspace click on the __View & Submit Jobs__ card on the Launcher tab.\n", "\n", "![job_ui_access](../_static/jobs_ui/jobs_ui_access.png)\n" ] }, { "cell_type": "markdown", "id": "9c8fe4fc-170f-498c-a796-4c95e558e739", "metadata": {}, "source": [ "## View Jobs\n", "\n", "The __View__ pane lists all the jobs submitted by the user. The top table shows only a few key fields. Users can click on any row to view detailed data for the selected job such as inputs, outputs generated, and errors produced if the job failed to complete successfully.\n", "\n", "Users can sort jobs by queued, start, and end time in ascending/descending order. Users may use the search bar to filter the job list down to jobs containing the user-provided string in any of the fields shown. \n", "\n", "![jobs_ui_overview](../_static/jobs_ui/jobs_ui_overview.png)\n", "\n", "### Job Status\n", "Job status may be different between the HySDS Figaro job-monitoring dashboard and the Jobs UI. Below is a mapping of status terms:\n", "```\n", "MAAP <- HySDS\n", "Accepted <- job-queued\n", "Running <- job-started\n", "Success <- job-completed\n", "Failed <- job-failed\n", "job-offline <- job-offline (your job will automatically re-queue when resources are available; this happens when a spot instance is given to another cloud customer while your job is running)\n", "job-revoked <- job-revoked (when a queued or running job is stopped before completion)\n", "\n", "HySDS state not valid/used in MAAP: job-deduped\n", "```\n" ] }, { "cell_type": "markdown", "id": "940e7588", "metadata": {}, "source": [ "## Viewing the Output of a Job\n", "\n", "Select your job then under the `Job Details` table, click the `Outputs` tab. The `Products` field provides the path to the product directory within the workspace.\n", "\n", "![job_result_2.png](../_static/faq_images/job_result_2.png)\n", "\n", "Navigating to the product directory for the selected job above shows the following:\n", "```\n", "(pangeo) root@workspacelbx1uqgeyqdei83w:~/my-private-bucket/dps_output/run-dps-test_ubuntu/delay10/2023/05/10/15/13/27/250000# pwd\n", "/projects/my-private-bucket/dps_output/run-dps-test_ubuntu/delay10/2023/05/10/15/13/27/250000\n", "\n", "(pangeo) root@workspacelbx1uqgeyqdei83w:~/my-private-bucket/dps_output/run-dps-test_ubuntu/delay10/2023/05/10/15/13/27/250000# ls\n", "_stderr.txt _stdout.txt output-2023-05-10T15:13:27.250000.context.json output-2023-05-10T15:13:27.250000.dataset.json output-2023-05-10T15:13:27.250000.met.json write-output.txt\n", "```" ] }, { "cell_type": "markdown", "id": "12d97e53-edc1-469f-8387-860571be53fc", "metadata": {}, "source": [ "## Submit a Job\n", "\n", "Users can submit individual jobs from the __Submit__ pane. The following are the minimum requirements for submitting a job:\n", "\n", "1. Select an Algorithm from the dropdown menu. Note: only registered algorithms will be shown.\n", "2. Provide a Job Tag that may then be used to easily search for and identify the submitted job. This is the same as the `identifier` field in the maap.py submitJob() function.\n", "3. Select the Resource to use for algorithm execution. This indicates the memory and disk space available.\n", "\n", ".. note::\n", "Experimental feature: The output data will be put into a folder named for your Algorithm and the Job Tag. You can use the same Job Tag on several jobs (e.g. in a batch) to group related output data in one place. In [maap.py submitJob](jobs_maappy.ipynb#Submit-a-Job) this is the `identifier` field when you use submitJob().\n", "\n", "![jobs_ui_submit](../_static/jobs_ui/jobs_ui_submit.png)\n", "\n", "Algorithms may contain additional inputs that users may have to provide.\n", "\n", "![jobs_ui_submit_detail](../_static/jobs_ui/jobs_ui_submit_detail.png)\n", "\n", "Once all inputs have been provided, the user may click __Submit Job__ to submit the job. If the job was submitted successfully, a toast will appear in the bottom right corner containing the unique job id. Please give the __Submit Job__ button a second to execute. \n", "\n", "![jobs_ui_submit_toast](../_static/jobs_ui/jobs_ui_submit_toast.png)\n", "\n", "If the job failed to submit, a toast will appear indicating the job failed to submit.\n", "\n", "![jobs_ui_failed_toast](../_static/jobs_ui/jobs_ui_failed_toast.png)" ] }, { "cell_type": "markdown", "id": "bc245e99-f531-4bcb-b504-c67d40352c29", "metadata": {}, "source": [ "## Generate Job Submission Command\n", "\n", "Users may fill out the job submission form and - instead of submitting the job - click the __Copy Jupyter Notebook Code__ button to copy the __maap-py__ job submission command to their clipboard to then paste it into a Jupyter notebook.\n", "\n", "![jobs_ui_copy_code](../_static/jobs_ui/jobs_ui_copy_code.png)" ] }, { "cell_type": "markdown", "id": "84cf6298", "metadata": {}, "source": [ "## Cancel Jobs\n", "\n", "Users may cancel queued or running jobs. Queued jobs that are cancelled will be deleted and no longer listed in the __View Jobs__ table. Running jobs that are cancelled will be stopped, but still viewable in the __View Jobs__ table. These jobs will have the status __job-revoked__.\n", "\n", ".. note::\n", "Jobs that are listed as queued in the UI, then cancelled, may remain in the View Jobs table after the cancel request was submitted. This is due to a delay within the DPS between job state change and report of job state change. In these cases, the cancelled job was not really queued but had already started.\n", "\n", "1. In the View Jobs table, click on the job you would like to cancel. If applicable (i.e. if the job is queued or running), there will be a \"Cancel Jobs\" button next to the Job Details header \n", "\n", "2. Click the cancel job button.\n", "\n", "![jobs_ui_revoke_job](../_static/jobs_ui/revoke_job.png)\n", "\n", "Once the cancel request is submitted, a toast will appear in the bottom right of the workspace indicating whether or not the request was successful.\n", "\n", "Successful submission:\n", "![jobs_ui_revoke_success](../_static/jobs_ui/revoke_successful.png)\n", "\n", "Unsuccessful submission:\n", "![jobs_ui_revoke_failed](../_static/jobs_ui/revoke_failed.png)\n", "\n", " " ] } ], "metadata": { "kernelspec": { "display_name": "Python 3.13.0 64-bit", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.0" }, "vscode": { "interpreter": { "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e" } } }, "nbformat": 4, "nbformat_minor": 5 }