12. Graphics Generation

Two Python plotting scripts are provided to generate plots from the FV3-LAM post-processed GRIB2 output over the CONUS for a number of variables, including:

  • 2-m temperature

  • 2-m dew point temperature

  • 10-m winds

  • 500 hPa heights, winds, and vorticity

  • 250 hPa winds

  • Accumulated precipitation

  • Composite reflectivity

  • Surface-based CAPE/CIN

  • Max/Min 2-5 km updraft helicity

  • Sea level pressure (SLP)

The Python scripts are located under ufs-srweather-app/ush/Python. The script plot_allvars.py plots the output from a single cycle within an experiment, while the script plot_allvars_diff.py plots the difference between the same cycle from two different experiments. If plotting the difference, the two experiments must be on the same domain and available for the same cycle starting date/time and forecast hours. Other parameters may differ (e.g., the experiments may use different physics suites).

12.1. Loading the Environment

To use the plotting scripts, the regional workflow environment, which includes the required scipy, matplotlib, pygrib, cartopy, and pillow packages, must be loaded. To activate the regional workflow, see Section 5.3.1, or use the following summary:

cd <path/to/ufs-srweather-app/ush/Python>
source ../../etc/lmod-setup.sh <platform>
module use ../../modulefiles
module load wflow_<platform>

where <platform> refers to a valid machine name (see Section 9.1). Then users should follow the instructions output by the console (e.g., conda activate regional_workflow).

12.2. Cartopy Shapefiles

The Python plotting scripts also require a path to the directory where the Cartopy Natural Earth shapefiles are located. The medium scale (1:50m) cultural and physical shapefiles are used to create coastlines and other geopolitical borders on the map. Cartopy provides the ‘background_img()’ method to add background images in a convenient way. The default scale (resolution) of background attributes in the Python scripts is 1:50m Natural Earth I with Shaded Relief and Water, which should be sufficient for most regional applications.

The full set of Cartopy shapefiles can be downloaded here. For convenience, the small subset of files required for these Python scripts can be obtained from the SRW Data Bucket. They are also available on all Level 1 platforms in the following locations:

Table 12.1 Cartopy Shapefile Locations for Level 1 Systems

Machine

File location

Cheyenne

/glade/p/ral/jntp/UFS_SRW_App/v2p1/NaturalEarth

Gaea

/lustre/f2/pdata/ncep/UFS_SRW_App/v2p1/NaturalEarth

Hera

/scratch2/BMC/det/UFS_SRW_App/v2p1/NaturalEarth

Jet

/mnt/lfs4/BMC/wrfruc/UFS_SRW_App/v2p1/NaturalEarth

NOAA Cloud

/contrib/EPIC/UFS_SRW_App/v2p1/NaturalEarth

Orion

/work/noaa/fv3-cam/UFS_SRW_App/v2p1/NaturalEarth

12.3. Running the Plotting Scripts

Before generating plots, it is helpful to change location to the directory containing the plotting scripts:

cd ufs-srweather-app/ush/Python

12.3.1. Plotting Output from One Experiment

To generate plots for a single cycle, the plot_allvars.py script must be called with the following command line arguments:

  1. Cycle date/time (CDATE) in YYYYMMDDHH format

  2. Starting forecast hour

  3. Ending forecast hour

  4. Forecast hour increment

  5. The top level of the experiment directory $EXPTDIR containing the post-processed data. The script will look for the data files in the directory $EXPTDIR/CDATE/postprd.

  6. The base directory CARTOPY_DIR of the cartopy shapefiles. The script will look for the shapefiles (*.shp) in the directory $CARTOPY_DIR/shapefiles/natural_earth/cultural. See Table 12.1 for the correct $CARTOPY_DIR locations on Level 1 systems.

  7. The name POST_OUTPUT_DOMAIN_NAME of the native grid used in the forecast

Note

If a forecast starts at 18h, this is considered the 0th forecast hour, so “starting forecast hour” should be 0, not 18.

An example of plotting output from a cycle generated using the sample experiment/workflow configuration in the config.community.yaml script (which uses the GFSv16 suite definition file) is as follows:

python plot_allvars.py 2019061518 0 12 6 /path-to/expt_dirs/test_community /path-to/NaturalEarth RRFS_CONUS_25km

The output files (in .png format) will be located in the directory $EXPTDIR/CDATE/postprd, where in this case $EXPTDIR is /path-to/expt_dirs/test_CONUS_25km_GFSv16 and $CDATE is 2019061518.

Note

The plotting script may generate a series of deprecation warnings, but users can safely ignore these. The script will generate the plots successfully despite them.

12.3.2. Plotting Differences from Two Experiments

To generate difference plots, the plot_allvars_diff.py script must be called with the following command line arguments:

  1. Cycle date/time (CDATE) in YYYYMMDDHH format

  2. Starting forecast hour

  3. Ending forecast hour

  4. Forecast hour increment

  5. The top level of the first experiment directory $EXPTDIR1 containing the first set of post-processed data. The script will look for the data files in the directory $EXPTDIR1/CDATE/postprd.

  6. The top level of the second experiment directory $EXPTDIR2 containing the second set of post-processed data. The script will look for the data files in the directory $EXPTDIR2/CDATE/postprd.

  7. The base directory CARTOPY_DIR of the cartopy shapefiles. The script will look for the shapefiles (*.shp) in the directory $CARTOPY_DIR/shapefiles/natural_earth/cultural.

  8. The name POST_OUTPUT_DOMAIN_NAME of the native grid used in the forecasts (this must be the same for the two forecasts)

An example of plotting differences from two experiments for the same date and predefined domain where one uses the FV3_GFS_v16 suite definition file (SDF) and one uses the FV3_RRFS_v1beta SDF is as follows:

python plot_allvars_diff.py 2019061518 0 12 6 /path-to/expt_dirs1/test_CONUS_3km_GFSv16 /path-to/expt_dirs2/test_CONUS_3km_RRFSv1beta /path-to/NaturalEarth RRFS_CONUS_25km

In this case, the output .png files will be located in the directory $EXPTDIR1/CDATE/postprd.

Note

The plotting script may generate a series of deprecation warnings, but users can safely ignore these. The script will generate the plots successfully despite them.

12.4. Submitting Plotting Scripts Through a Batch System

If users plan to create plots of multiple forecast lead times and forecast variables, then they may need to submit the Python scripts to the batch system. Sample scripts are provided for use on a platform such as Hera that uses the Slurm job scheduler: sq_job.sh and sq_job_diff.sh. Equivalent sample scripts are provided for use on a platform such as Cheyenne that uses PBS as the job scheduler: qsub_job.sh and qsub_job_diff.sh. Examples of these scripts are located under ufs-srweather-app/ush/Python and can be used as a starting point to create a batch script for the user’s specific platform/job scheduler.

At a minimum, the account should be set appropriately prior to job submission:

#SBATCH --account=<account_name>

Depending on the platform, users may also need to adjust the settings to use the correct Python environment and path to the shapefiles.

When working with these batch scripts, several environment variables must be set prior to submission. If plotting output from a single cycle, the variables to set are $HOMEdir and $EXPTDIR. If the user’s login shell is bash, these variables can be set as follows:

export HOMEdir=/path-to/ufs-srweather-app
export EXPTDIR=/path-to/experiment/directory

If the user’s login shell is csh/tcsh, they can be set as follows:

setenv HOMEdir /path-to/ufs-srweather-app
setenv EXPTDIR /path-to/experiment/directory

If plotting the difference between the same cycle from two different experiments, the variables to set are $HOMEdir, $EXPTDIR1, and $EXPTDIR2. If the user’s login shell is bash, these variables can be set as follows:

export HOMEdir=/path-to/ufs-srweather-app
export EXPTDIR1=/path-to/experiment/directory1
export EXPTDIR2=/path-to/experiment/directory2

If the user’s login shell is csh/tcsh, they can be set as follows:

setenv HOMEdir /path-to/ufs-srweather-app
setenv EXPTDIR1 /path-to/experiment/directory1
setenv EXPTDIR2 /path-to/experiment/directory2

In addition, the variables CDATE, FCST_START, FCST_END, and FCST_INC in the batch scripts can be modified depending on the user’s needs. By default, CDATE is set as follows in the batch scripts:

export CDATE=${DATE_FIRST_CYCL}

This sets CDATE to the first cycle in the set of cycles that the experiment has run. If the experiment contains multiple cycles and the user wants to plot output from a cycle other than the very first one, CDATE in the batch scripts will have to be set to the specific YYYYMMDDHH value for that cycle. Also, to plot hourly forecast output, FCST_INC should be set to 1; to plot only a subset of the output hours, FCST_START, FCST_END, and FCST_INC must be set accordingly. For example, to generate plots for every 3rd forecast hour starting with forecast hour 6 and ending with the last forecast hour, use:

export FCST_START=6
export FCST_END=${FCST_LEN_HRS}
export FCST_INC=3

The scripts must be submitted using the command appropriate for the job scheduler on the user’s platform. For example, on Hera, sq_job.sh can be submitted as follows:

sbatch sq_job.sh

On Cheyenne, qsub_job.sh can be submitted as follows:

qsub qsub_job.sh