12. Graphics Generation

Two Python plotting scripts are provided to generate plots from the FV3-LAM post-processed GRIB2 output over the CONUS for a number of variables, including:

  • 2-m temperature

  • 2-m dew point temperature

  • 10-m winds

  • 500 hPa heights, winds, and vorticity

  • 250 hPa winds

  • Accumulated precipitation

  • Composite reflectivity

  • Surface-based CAPE/CIN

  • Max/Min 2-5 km updraft helicity

  • Sea level pressure (SLP)

The Python scripts are located under ufs-srweather-app/regional_workflow/ush/Python. The script plot_allvars.py plots the output from a single cycle within an experiment, while the script plot_allvars_diff.py plots the difference between the same cycle from two different experiments (e.g., the experiments may differ in some aspect such as the physics suite used). If plotting the difference, the two experiments must be on the same domain and available for the same cycle starting date/time and forecast hours.

12.1. Loading the Environment

To use the plotting scripts, the appropriate environment must be loaded. The scripts require Python 3 with the scipy, matplotlib, pygrib, cartopy, and pillow packages. This Python environment has already been set up on Level 1 platforms and can be activated as follows:

On Cheyenne:

module load ncarenv
ncar_pylib /glade/p/ral/jntp/UFS_SRW_app/ncar_pylib/python_graphics

On Hera and Jet:

module use -a /contrib/miniconda3/modulefiles
module load miniconda3
conda activate pygraf

On Orion:

module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles
module load miniconda3
conda activate pygraf

On Gaea:

module use /lustre/f2/pdata/esrl/gsd/contrib/modulefiles
module load miniconda3/4.8.3-regional-workflow

On NOAA Cloud:

module use /contrib/GST/miniconda3/modulefiles
module load miniconda3/4.10.3
conda activate regional_workflow

Note

If using one of the batch submission scripts described below, the user does not need to manually load an environment because the scripts perform this task.

12.2. Cartopy Shape Files

The Python plotting scripts also require a path to the directory where the Cartopy Natural Earth shape files are located. The medium scale (1:50m) cultural and physical shapefiles are used to create coastlines and other geopolitical borders on the map. Cartopy provides the ‘background_img()’ method to add background images in a convenient way. The default scale (resolution) of background attributes in the Python scripts is 1:50m Natural Earth I with Shaded Relief and Water, which should be sufficient for most regional applications.

The full set of Cartopy shape files can be downloaded here. For convenience, the small subset of files required for these Python scripts can be obtained from the SRW Data Bucket. They are also available on all Level 1 platforms in the following locations:

On Cheyenne:

/glade/p/ral/jntp/UFS_SRW_App/v2p0/NaturalEarth

On Hera:

/scratch2/BMC/det/UFS_SRW_App/v2p0/NaturalEarth

On Jet:

/mnt/lfs4/BMC/wrfruc/UFS_SRW_App/v2p0/NaturalEarth

On Orion:

/work/noaa/fv3-cam/UFS_SRW_App/v2p0/NaturalEarth

On Gaea:

/lustre/f2/pdata/ncep/UFS_SRW_App/v2p0/NaturalEarth

On NOAA Cloud:

/contrib/EPIC/UFS_SRW_App/v2p0/NaturalEarth

12.3. Running the Plotting Scripts

Before generating plots, it is helpful to change location to the directory containing the plotting scripts:

cd ufs-srweather-app/regional_workflow/ush/Python

12.3.1. Plotting Output from One Experiment

To generate plots for a single cycle, the plot_allvars.py script must be called with the following command line arguments:

  1. Cycle date/time (CDATE) in YYYYMMDDHH format

  2. Starting forecast hour

  3. Ending forecast hour

  4. Forecast hour increment

  5. The top level of the experiment directory EXPTDIR containing the post-processed data. The script will look for the data files in the directory $EXPTDIR/CDATE/postprd.

  6. The base directory CARTOPY_DIR of the cartopy shapefiles. The script will look for the shape files (*.shp) in the directory $CARTOPY_DIR/shapefiles/natural_earth/cultural.

  7. The name POST_OUTPUT_DOMAIN_NAME of the native grid used in the forecast

Note

If a forecast starts at 18h, this is considered the 0th forecast hour, so “starting forecast hour” should be 0, not 18.

An example of plotting output from a cycle generated using the sample experiment/workflow configuration in the config.community.sh script (which uses the GFSv16 suite definition file) is as follows:

python plot_allvars.py 2019061500 0 12 6 /path-to/expt_dirs/test_CONUS_25km_GFSv16 /path-to/NaturalEarth RRFS_CONUS_25km

The output files (in .png format) will be located in the directory $EXPTDIR/CDATE/postprd, where in this case $EXPTDIR is /path-to/expt_dirs/test_CONUS_25km_GFSv16 and $CDATE is 2019061518.

12.3.2. Plotting Differences from Two Experiments

To generate difference plots, the plot_allvars_diff.py script must be called with the following command line arguments:

  1. Cycle date/time (CDATE) in YYYYMMDDHH format

  2. Starting forecast hour

  3. Ending forecast hour

  4. Forecast hour increment

  5. The top level of the first experiment directory EXPTDIR1 containing the first set of post-processed data. The script will look for the data files in the directory $EXPTDIR1/CDATE/postprd.

  6. The top level of the second experiment directory EXPTDIR2 containing the second set of post-processed data. The script will look for the data files in the directory $EXPTDIR2/CDATE/postprd.

  7. The base directory CARTOPY_DIR of the cartopy shapefiles. The script will look for the shape files (*.shp) in the directory $CARTOPY_DIR/shapefiles/natural_earth/cultural.

  8. The name POST_OUTPUT_DOMAIN_NAME of the native grid used in the forecasts (this must be the same for the two forecasts)

An example of plotting differences from two experiments for the same date and predefined domain where one uses the “FV3_GFS_v16” suite definition file (SDF) and one using the “FV3_RRFS_v1beta” SDF is as follows:

python plot_allvars_diff.py 2019061518 0 12 6 /path-to/expt_dirs1/test_CONUS_3km_GFSv16 /path-to/expt_dirs2/test_CONUS_3km_RRFSv1beta /path-to/NaturalEarth RRFS_CONUS_25km

12.4. Submitting Plotting Scripts Through a Batch System

If users plan to create plots of multiple forecast lead times and forecast variables, then they may need to submit the Python scripts to the batch system. Sample scripts are provided for use on a platform such as Hera that uses the Slurm job scheduler: sq_job.sh and sq_job_diff.sh. Equivalent sample scripts are provided for use on a platform such as Cheyenne that uses PBS as the job scheduler: qsub_job.sh and qsub_job_diff.sh. Examples of these scripts are located under ufs-srweather-app/regional_workflow/ush/Python and can be used as a starting point to create a batch script for the user’s specific platform/job scheduler.

At a minimum, the account should be set appropriately prior to job submission:

#SBATCH --account=<account_name>

Depending on the platform, users may also need to adjust the settings to use the correct Python environment and path to the shape files.

When working with these batch scripts, several environment variables must be set prior to submission. If plotting output from a single cycle, the variables to set are HOMErrfs and EXPTDIR. If the user’s login shell is bash, these variables can be set as follows:

export HOMErrfs=/path-to/ufs-srweather-app/regional_workflow
export EXPTDIR=/path-to/experiment/directory

If the user’s login shell is csh/tcsh, they can be set as follows:

setenv HOMErrfs /path-to/ufs-srweather-app/regional_workflow
setenv EXPTDIR /path-to/experiment/directory

If plotting the difference between the same cycle from two different experiments, the variables to set are HOMErrfs, EXPTDIR1, and EXPTDIR2. If the user’s login shell is bash, these variables can be set as follows:

export HOMErrfs=/path-to/ufs-srweather-app/regional_workflow
export EXPTDIR1=/path-to/experiment/directory1
export EXPTDIR2=/path-to/experiment/directory2

If the user’s login shell is csh/tcsh, they can be set as follows:

setenv HOMErrfs /path-to/ufs-srweather-app/regional_workflow
setenv EXPTDIR1 /path-to/experiment/directory1
setenv EXPTDIR2 /path-to/experiment/directory2

In addition, the variables CDATE, FCST_START, FCST_END, and FCST_INC in the batch scripts can be modified depending on the user’s needs. By default, CDATE is set as follows in the batch scripts:

export CDATE=${DATE_FIRST_CYCL}${CYCL_HRS}

This sets CDATE to the first cycle in the set of cycles that the experiment has run. If the experiment contains multiple cycles and the user wants to plot output from a cycle other than the very first one, CDATE in the batch scripts will have to be set to the specific YYYYMMDDHH value for that cycle. Also, to plot hourly forecast output, FCST_INC should be set to 1; to plot only a subset of the output hours, FCST_START, FCST_END, and FCST_INC must be set accordingly. For example, to generate plots for every 3rd forecast hour starting with forecast hour 6 and ending with the last forecast hour, use:

export FCST_START=6
export FCST_END=${FCST_LEN_HRS}
export FCST_INC=3

The scripts must be submitted using the command appropriate for the job scheduler on the user’s platform. For example, on Hera, sq_job.sh can be submitted as follows:

sbatch sq_job.sh

On Cheyenne, qsub_job.sh can be submitted as follows:

qsub qsub_job.sh