NERSCPowering Scientific Discovery Since 1974

Modules Software Environment

NERSC uses the module utility to manage nearly all software. There are two huge advantages of the module approach:

  1. NERSC can provide many different versions and/or installations of a single software package on a given machine, including a default version as well as several older and newer versions; and
  2. Users can easily switch to different versions or installations without having to explicitly specify different paths.  With modules, the MANPATH and related environment variables are automatically managed. Users simply ``load'' and ``unload'' modules to control their environment.

The module utility consists of two parts: the module command itself and the modulefiles on which it operates. 

Module Command

module help

To get a usage list of module options type the following (the listing has been abbreviated to only those commands discussed on this page) :

% module help  

Available Commands and Usage:

+ add|load modulefile [modulefile ...]
+ rm|unload modulefile [modulefile ...]
+ switch modulefile1 modulefile2
+ display modulefile [modulefile ...]
+ avail path [path]
+ list
+ help modulefile [modulefile ...]

To get help on a specific module use:

 % module help [modulefile]


module list

% module list

This lists all the modulefiles that are currently loaded into your environment.

module avail

% module avail

This option lists all the modulefiles that are available to be loaded. Notice that many of them have version numbers associated with them and that where there is more than one version, one is labeled as the default. You can also restrict this command to a single package; for example,

module avail netcdf

This option will list all the module files that start with "netcdf".  

It is often more useful to use "-S" flag along with the module avail command, to return all module file names that have the substring of "netcdf" in it.  For example, the following command will return all netcdf and cray-netcdf modules.

module avail -S netcdf


module display

 % module display [modulefile]

Use this command to see exactly what a given modulefile will do to your environment, such as what will be added to the PATH, MANPATH, etc., environment variables.  This is synonymous with

 % module show [modulefile]


module load

% module load [modulefile1] [modulefile2] ...

This command adds one or more modulefiles to your current environment. It does so silently, unless there is a problem with a modulefile. If you load the generic name of a module, you will get the default version. To load a specific version, load the modulefile using its full specification.  For example,

 % module load visit

will do the same thing as

% module load visit//2.1.2


module unload

% module unload [modulefile]

This removes the listed module from the user's current environment. Modulefiles can be removed in any order.  Note that this command will fail silently if the modulefile you specify is not already loaded.

module switch

% module switch [modulefile_old] [modulefile_new]

This command demonstrates the true advantage of using modules. Different versions of entire software packages can be replaced with a single module command.  

This is synonymous with

 % module swap [modulefile_old] [modulefile_new]

Note that the following command would be completely correct and appropriate if, before issuing it, you had issued a "module load" command for a non-default version of the VisIt software and you wish to switch to the default version:

module swap visit visit

Please refer to "module" man page for more details.

Loading Modules into Your Default Environment

You can modify your environment so that certain modules are loaded whenever you log in. Put your changes in one of the following files, depending on your shell.

  • .cshrc.ext or .tcshrc.ext
  • .bashrc.ext

Here is an example of a .cshrc.ext file with commands to load modules. Notice that, modules specific to a platform are put inside of if-then blocks.

if ($NERSC_HOST == "cori") then
# Replace the following line with personal settings for Cori
module load fftw
endif

Saving and restoring the programming environment on Cori and Edison

The default programming environment on Cray systems changes over time. Usually the Cray Developer Toolkit (CDT) version is upgraded about every six months on Cori and Edison so that users can get benefit from the enhanced software features and performance. However, if you need to compile a code with a specific version of CDT which is not  the default version anymore, or if your code works best with a mix of default and non-default software modules and you want to recompile with the same set of modules, then you can use the Cray provided cdt modules or the module snapshot feature to restore the compilation environment where your code was built previously. Additionally, you can use Shifter or Docker images to preserve your programming environment and other environments as well. 

Module Snapshot

The module subcommand "snapshot" is available starting from modules/3.2.10.5. The command captures the currently loaded modules and saves the list into a file for later restore. Here is how to use it:

To save the current module environment into a file, do

module snapshot -f filename

The file can be provided with a full path or just with a file name. If a full path is not given, then the snapshot will be saved in your ${HOME}/.module_snapshots directory. An environment variable, MODULE_SNAPSHOT_DIR can be used to use an alternative location. 

To restore later, do

module restore filename

If a file is not provided with a full path, then the $MODULE_SNAPSHOT_DIR will be searched first, followed by $HOME/.module_snapshots. 

Here is the demo: 

save modules

restore modules

 

Note, you may see the module restore command fails sometime, depending on the modules loaded in the current environment and the ones to be loaded in the module snapshot file. If this occurs, try a fresh login session before executing the module restore command. You may also see errors with the module restore command if the software modules in your snapshot do not exists anymore. Please report any problems with the module snapshot feature to consult@nersc.gov.   

CDT modules

The cdt modules are provided by Cray to do a clean switch between different CDT modules. Type module avail cdt to see the available cdt modules.

cdt switch

  

Using shifter 

Shifter is a software package that allows user-created images to run at NERSC. Using Shifter you can create an image with your desired operating system and software stacks and dependencies. Shifter is especially useful if your workload requires performance reproducibility over time because it is not only preserve the programming environments but also the operating systems. You can find more details about how to build Shifter image and use it on our systems at here. 

Install your own customized modules

You can create and install your own modules for your convenience or for sharing software among collaborators. The module definition files can be in a project directory, your home directory, or any available file system.  Make sure the UNIX file permissions grant access to all users who want to use the  software. (Note: do not give write permissions in your home directory to anyone else!)

An example of such a modulefile "1.2.7" for the customized module "myzlib" is located at /global/project/projectdirs/mpccc/usg/modulefiles/hopper/myzlib.  This modulefile includes a nice feature to detect the current compiler environment on Cray systems (such as pgi or intel, with the definifition of $LCcompiler), and adds the corresponding include and libaray paths automatically (with the definition of $XXX_POST_LINK_OPTS and $XXX_INCLUDE_OPTS) to the compiler wrappers.  To use this module, just do the following:

% module use /global/project/projectdirs/modulefiles/hopper
% module load myzlib/1.2.7

Note that the "module use" command adds this new directory before other module search paths (defined as $MODULEPATH), so modules defined in this custom directory will have precedence if there are other modules with the same name in the module search paths.  If you prefer to have the new directory added at the end of $MODULEPATH, use "module use -a" instead of "module use" in the above command. 

Please contact NERSC consultants if you need further help in creating your own modules.