You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
@@ -9,116 +9,161 @@ Running I-WRF on NCAR HPCs with Land Use/Land Cover Change Data
9
9
Overview
10
10
========
11
11
12
-
Update section with an overview of the following instructions.
12
+
The following instructions can be used to run METplus verification for the Land Use/Land Cover Change (LULC) use case of the I-WRF weather simulation framework from the National Center for Atmospheric Research (NCAR) on NCAR High Performance Computing (HPC) platforms. The steps below configure and execute the METplus verification framework using containerized applications with Apptainer (formerly Singularity) to analyze I-WRF LULC simulation output.
13
+
14
+
The LULC use case examines how changes in land surface characteristics impact weather patterns and atmospheric dynamics through idealized modeling scenarios. METplus provides statistical analysis and evaluation tools to quantify these meteorological impacts, making this verification workflow valuable for studies in urban meteorology, land-atmosphere interactions, and climate change impact assessment.
15
+
16
+
NCAR HPC systems provide the computational resources needed for efficient METplus processing of large I-WRF datasets. This exercise uses containerized METplus applications, which simplifies the setup work needed to run the verification and ensures consistency across different HPC environments.
17
+
18
+
It is recommended that you follow the instructions in each section in the order presented to avoid encountering issues during the process. This guide assumes that I-WRF LULC simulations have already been completed and focuses specifically on the METplus verification workflow.
19
+
13
20
14
21
Prepare to Use NCAR HPCs
15
22
========================
16
23
17
-
Update section with instructions on how to set up the system to run i-wrf.
24
+
To get started with running METplus verification on NCAR HPC systems, you will need access to one of the supported platforms (Derecho, Casper, or other NCAR computing resources). If you do not already have access, you will need to request an account through the NCAR Computing and Information Systems Laboratory (CISL).
25
+
26
+
NCAR HPC access is typically provided to researchers affiliated with NCAR, university collaborators, or those with approved allocations. If you need to request access:
18
27
19
-
Examples include:
28
+
* Visit the NCAR CISL user registration page to create an account
29
+
* Provide information about your research project and institutional affiliation
30
+
* Wait for account approval, which typically takes 1-2 business days
31
+
* Once approved, you can log in to the HPC systems using SSH
20
32
21
-
* Gaining access to the system
22
-
* Creating instances
23
-
* Managing instances.
33
+
Most NCAR HPC systems use a batch job scheduling system for computational work. While this exercise can be run interactively for demonstration purposes, production runs should be submitted through the job scheduler. Refer to the NCAR HPC documentation for specific guidance on your target system.
34
+
Once you have access to an NCAR HPC system, you can log in using SSH from your local machine or through NCAR's web-based interfaces where available.
24
35
25
36
Preparing the Environment
26
37
=========================
27
38
28
-
Load the apptainer module::
39
+
With your NCAR HPC account active and you logged in to the system, you can now set up the environment and create the necessary directories to run METplus verification. You will only need to perform these steps once per system, as the directory structure and configuration will remain available for future use.
40
+
41
+
The following sections instruct you to issue numerous Linux commands in your shell. The commands in each section can be copied and pasted into your shell terminal.
42
+
43
+
Load Required Modules
44
+
---------------------
45
+
46
+
NCAR HPC systems use environment modules to manage software. Load the Apptainer module which provides the containerization software needed to run METplus::
29
47
30
48
module load apptainer
31
49
32
-
Create a working directory in the scratch area::
50
+
Define Environment Variables
51
+
----------------------------
33
52
34
-
IWRF_WORK_DIR=${SCRATCH}/iwrf_work
53
+
We will be using environment variables throughout this exercise to ensure consistent file paths and resource names. Copy and paste the definitions below into your shell before proceeding::
35
54
36
-
Create a directory to store the METplus output::
55
+
IWRF_WORK_DIR=${SCRATCH}/iwrf_work
56
+
LOCAL_OUTPUT_DIR=${IWRF_WORK_DIR}/metplus_out
57
+
export APPTAINER_TMPDIR=${TMPDIR}
37
58
38
-
LOCAL_OUTPUT_DIR=${IWRF_WORK_DIR}/metplus_out
39
-
mkdir -p ${LOCAL_OUTPUT_DIR}
59
+
Any time you open a new shell session on the HPC system, you will need to reload the apptainer module and redefine these variables before executing the commands that follow.
40
60
41
-
Create a directory to store temporary Apptainer files
42
-
($TMPDIR is set automatically for all users on NCAR HPC machines)::
61
+
Create Working Directories
62
+
--------------------------
43
63
44
-
export APPTAINER_TMPDIR=${TMPDIR}
45
-
mkdir -p ${APPTAINER_TMPDIR}
64
+
The METplus verification process requires specific directory structures to organize input data, configuration files, and output results. Create the main working directory in your scratch space::
46
65
47
-
Clone the I-WRF GitHub repository to get the configuration files::
Create a directory to store the METplus verification output::
50
69
51
-
Install Docker and Pull Docker Objects
52
-
======================================
70
+
mkdir -p ${LOCAL_OUTPUT_DIR}
53
71
54
-
Update section with instructions on how to install Docker and pull docker images needed.
72
+
Create a directory for temporary Apptainer files. The $TMPDIR variable is automatically set on NCAR HPC systems to an appropriate temporary storage location::
55
73
56
-
Download Data for WRF
57
-
=====================
74
+
mkdir -p ${APPTAINER_TMPDIR}
58
75
59
-
Update section with instructions on how to download the data needed to run WRF.
76
+
Download Configuration Files
77
+
----------------------------
60
78
61
-
Run WRF
62
-
=======
79
+
METplus requires configuration files to direct its verification behavior. These are available in the I-WRF GitHub repository. Clone the repository to access the LULC use case configuration::
63
80
64
-
Update section with instructions on how to run WRF.
This creates a local copy of all I-WRF configuration files, including the METplus settings needed for the LULC verification workflow.
84
+
85
+
Pull Apptainer Objects
86
+
======================
68
87
69
-
Pull the METplus and input data containers from DockerHub. ::
88
+
As mentioned above, the METplus software is provided as a containerized image that will run using Apptainer on your NCAR HPC system. Apptainer (formerly Singularity) is the preferred containerization technology on HPC systems, as it provides secure container execution without requiring root privileges. Unlike cloud environments that use Docker directly, NCAR HPC systems use Apptainer to run containerized applications.
89
+
90
+
The METplus image contains all the necessary software and dependencies to perform verification of I-WRF LULC simulation output. You can "pull" (download) the METplus image from the container registry to your HPC system's storage.
91
+
92
+
Get the METplus and Data Container Images
93
+
-----------------------------------------
94
+
95
+
You must pull the METplus software container and the input data containers that contain the observational and WRF simulation data for the LULC use case::
These commands download three container images: the METplus software, the observational data, and the WRF simulation data. The process may take several minutes depending on your network connection.
102
+
103
+
Run METplus
104
+
===========
105
+
106
+
After the container images have been downloaded, you can run the METplus verification to compare the I-WRF LULC simulation results against observational data and generate statistical verification results and visualization plots. This process involves configuring the data bindings and executing the verification workflow for two meteorological variables.
107
+
108
+
Configure Container Data Bindings
109
+
---------------------------------
110
+
111
+
METplus requires access to input data, configuration files, and output directories. Apptainer uses bind mounts to make local directories and container images available inside the running container. Set up the environment variables that define these data bindings.
112
+
113
+
First, define the local directory paths for configuration and visualization scripts::
Next, configure the Apptainer bind mounts. This environment variable tells Apptainer how to map local directories and container images to paths inside the running container::
Execute the run_metplus.py command inside the container to run the use case
122
+
This configuration provides the container with access to:
123
+
124
+
* Observational data from the ``data-lulc-input-obs.sif`` container image at ``/data/input/obs``
125
+
* WRF simulation data from the ``data-lulc-input-wrf.sif`` container image at ``/data/input/wrf``
126
+
* METplus configuration files from the I-WRF repository at ``/config``
127
+
* Visualization script files for generating plots at ``/plot_scripts``
128
+
* Output directory for writing verification results at ``/data/output``
129
+
* Temporary directory for Apptainer operations
105
130
106
-
* Accumulated Precipitation::
131
+
Execute METplus Verification
132
+
----------------------------
133
+
134
+
The LULC use case includes verification for two meteorological variables: accumulated precipitation and radar reflectivity. Each verification is run separately using its own METplus configuration file.
This process compares simulated precipitation accumulation against observational data and generates statistical metrics. Progress information is displayed while the verification is performed.
This process evaluates the model's ability to simulate radar reflectivity patterns compared to observed radar data.
147
+
148
+
Both verification processes use GridStat, which computes grid-to-grid verification statistics. The tools generate comprehensive statistical output including bias, correlation, and skill scores that quantify the model's performance.
149
+
150
+
Verify Output Generation
151
+
------------------------
152
+
153
+
After both METplus runs complete successfully, you can verify that the output files were created properly.
154
+
155
+
Check that the GridStat verification output was generated::
115
156
116
157
ls ${LOCAL_OUTPUT_DIR}/grid_stat/* -1
117
158
118
-
Check that the METplotpy plots were created locally::
159
+
This should show directories containing statistical output files in text format that can be viewed and analyzed.
160
+
161
+
Check that the METplotpy visualization plots were created locally::
119
162
120
163
ls ${LOCAL_OUTPUT_DIR}/met_plot/*/*.png -1
121
164
165
+
This should display a list of PNG image files containing plots and graphics that visualize the verification results. These plots provide graphical representations of the statistical comparisons between the I-WRF LULC simulations and observational data.
0 commit comments