10. Graphics Generation

Two Python plotting scripts are provided to generate plots from the FV3-LAM post-processed GRIB2 output over the CONUS for a number of variables, including:

  • 2-m temperature

  • 2-m dew point temperature

  • 10-m winds

  • 500 hPa heights, winds, and vorticity

  • 250 hPa winds

  • Accumulated precipitation

  • Composite reflectivity

  • Surface-based CAPE/CIN

  • Max/Min 2-5 km updraft helicity

  • Sea level pressure (SLP)

The Python scripts are located under ufs-srweather-app/regional_workflow/ush/Python. The script plot_allvars.py plots the output from a single cycle within an experiment, while the script plot_allvars_diff.py plots the difference between the same cycle from two different experiments (e.g. the experiments may differ in some aspect such as the physics suite used). If plotting the difference, the two experiments must be on the same domain and available for the same cycle starting date/time and forecast hours.

The Python scripts require a cycle starting date/time in YYYYMMDDHH format, a starting forecast hour, an ending forecast hour, a forecast hour increment, the paths to one or two experiment directories, and a path to the directory where the Cartopy Natural Earth shape files are located. The full set of Cartopy shape files can be downloaded at https://www.naturalearthdata.com/downloads/. For convenience, the small subset of files required for these Python scripts can be obtained from the EMC ftp data repository or from AWS cloud storage. In addition, the Cartopy shape files are available on a number of Level 1 platforms in the following locations:

On Cheyenne:

/glade/p/ral/jntp/UFS_SRW_app/tools/NaturalEarth

On Hera:

/scratch2/BMC/det/UFS_SRW_app/v1p0/fix_files/NaturalEarth

On Jet:

/lfs4/BMC/wrfruc/FV3-LAM/NaturalEarth

On Orion:

/work/noaa/gsd-fv3-dev/UFS_SRW_App/v1p0/fix_files/NaturalEarth

On Gaea:

/lustre/f2/pdata/esrl/gsd/ufs/NaturalEarth

The medium scale (1:50m) cultural and physical shapefiles are used to create coastlines and other geopolitical borders on the map. Cartopy provides the ‘background_img()’ method to add background images in a convenient way. The default scale (resolution) of background attributes in the Python scripts is 1:50m Natural Earth I with Shaded Relief and Water, which should be sufficient for most regional applications.

The appropriate environment must be loaded to run the scripts, which require Python 3 with the scipy, matplotlib, pygrib, cartopy, and pillow packages. This Python environment has already been set up on Level 1 platforms and can be activated as follows:

On Cheyenne:

module load ncarenv
ncar_pylib /glade/p/ral/jntp/UFS_SRW_app/ncar_pylib/python_graphics

On Hera and Jet:

module use -a /contrib/miniconda3/modulefiles
module load miniconda3
conda activate pygraf

On Orion:

module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles
module load miniconda3
conda activate pygraf

On Gaea:

module use /lustre/f2/pdata/esrl/gsd/contrib/modulefiles
module load miniconda3/4.8.3-regional-workflow

Note

If using one of the batch submission scripts described below, the user does not need to manually load an environment because the scripts perform this task.

10.1. Plotting output from one experiment

Before generating plots, it is convenient to change location to the directory containing the plotting scripts:

cd ufs-srweather-app/regional_workflow/ush/Python

To generate plots for a single cycle, the plot_allvars.py script must be called with the following six command line arguments:

  1. Cycle date/time (CDATE) in YYYYMMDDHH format

  2. Starting forecast hour

  3. Ending forecast hour

  4. Forecast hour increment

  5. The top level of the experiment directory EXPTDIR containing the post-processed data. The script will look for the data files in the directory EXPTDIR/CDATE/postprd.

  6. The base directory CARTOPY_DIR of the cartopy shapefiles. The script will look for the shape files (*.shp) in the directory CARTOPY_DIR/shapefiles/natural_earth/cultural.

An example of plotting output from a cycle generated using the sample experiment/workflow configuration in the config.community.sh script (which uses the GFSv15p2 suite definition file) is as follows:

python plot_allvars.py 2019061500 6 48 6 /path-to/expt_dirs/test_CONUS_25km_GFSv15p2 /path-to/NaturalEarth

The output files (in .png format) will be located in the directory EXPTDIR/CDATE/postprd, where in this case EXPTDIR is /path-to/expt_dirs/test_CONUS_25km_GFSv15p2 and CDATE is 2019061500.

10.2. Plotting differences from two experiments

To generate difference plots, the plot_allvars_diff.py script must be called with the following seven command line arguments:

  1. Cycle date/time (CDATE) in YYYYMMDDHH format

  2. Starting forecast hour

  3. Ending forecast hour

  4. Forecast hour increment

  5. The top level of the first experiment directory EXPTDIR1 containing the first set of post-processed data. The script will look for the data files in the directory EXPTDIR1/CDATE/postprd.

  6. The top level of the first experiment directory EXPTDIR2 containing the second set of post-processed data. The script will look for the data files in the directory EXPTDIR2/CDATE/postprd.

  7. The base directory CARTOPY_DIR of the cartopy shapefiles. The script will look for the shape files (*.shp) in the directory CARTOPY_DIR/shapefiles/natural_earth/cultural.

An example of plotting differences from two experiments for the same date and predefined domain where one uses the “FV3_GFS_v15p2” suite definition file (SDF) and one using the “FV3_RRFS_v1alpha” SDF is as follows:

python plot_allvars_diff.py 2019061518 6 18 3 /path-to/expt_dirs1/test_CONUS_3km_GFSv15p2 /path-to/expt_dirs2/test_CONUS_3km_RRFSv1alpha /path-to/NaturalEarth

In this case, the output png files will be located in the directory EXPTDIR1/CDATE/postprd.

10.3. Submitting plotting scripts through a batch system

If the Python scripts are being used to create plots of multiple forecast lead times and forecast variables, then you may need to submit them to the batch system. Example scripts are provided called sq_job.sh and sq_job_diff.sh for use on a platform such as Hera that uses the Slurm job scheduler or qsub_job.sh and qsub_job_diff.sh for use on a platform such as Cheyenne that uses PBS as the job scheduler. Examples of these scripts are located under ufs-srweather-app/regional_workflow/ush/Python and can be used as a starting point to create a batch script for your platform/job scheduler of use.

At a minimum, the account should be set appropriately prior to job submission:

#SBATCH --account=an_account

Depending on the platform you are running on, you may also need to adjust the settings to use the correct Python environment and path to the shape files.

When using these batch scripts, several environment variables must be set prior to submission. If plotting output from a single cycle, the variables to set are HOMErrfs and EXPTDIR. In this case, if the user’s login shell is csh/tcsh, these variables can be set as follows:

setenv HOMErrfs /path-to/ufs-srweather-app/regional_workflow
setenv EXPTDIR /path-to/experiment/directory

If the user’s login shell is bash, they can be set as follows:

export HOMErrfs=/path-to/ufs-srweather-app/regional_workflow
export EXPTDIR=/path-to/experiment/directory

If plotting the difference between the same cycle from two different experiments, the variables to set are HOMErrfs, EXPTDIR1. and EXPTDIR2. In this case, if the user’s login shell is csh/tcsh, these variables can be set as follows:

setenv HOMErrfs /path-to/ufs-srweather-app/regional_workflow
setenv EXPTDIR1 /path-to/experiment/directory1
setenv EXPTDIR2 /path-to/experiment/directory2

If the user’s login shell is bash, they can be set as follows:

export HOMErrfs=/path-to/ufs-srweather-app/regional_workflow
export EXPTDIR1=/path-to/experiment/directory1
export EXPTDIR2=/path-to/experiment/directory2

In addition, the variables CDATE, FCST_START, FCST_END, and FCST_INC in the batch scripts can be modified depending on the user’s needs. By default, CDATE is set as follows in the batch scripts:

export CDATE=${DATE_FIRST_CYCL}${CYCL_HRS}

This sets CDATE to the first cycle in the set of cycles that the experiment has run. If the experiment contains multiple cycles and the user wants to plot output from a cycle other than the very first one, CDATE in the batch scripts will have to be set to the specific YYYYMMDDHH value for that cycle. Also, to plot hourly forecast output, FCST_INC should be set to 1; to plot only a subset of the output hours, FCST_START, FCST_END, and FCST_INC must be set accordingly, e.g. to generate plots for every 6th forecast hour starting with forecast hour 6 and ending with the last forecast hour, use

export FCST_START=6
export FCST_END=${FCST_LEN_HRS}
export FCST_INC=6

The scripts must be submitted using the command appropriate for the job scheduler used on your platform. For example, on Hera, sq_job.sh can be submitted as follows:

sbatch sq_job.sh

On Cheyenne, qsub_job.sh can be submitted as follows:

qsub qsub_job.sh