NERSCPowering Scientific Discovery Since 1974

NWChem

Description

NWChem is a computational chemistry package designed to run on high-performance parallel supercomputers as well as conventional workstation clusters. It aims to be scalable both in its ability to treat large problems efficiently, and in its usage of available parallel computing resources.

How to access NWChem

NERSC uses modules to manage access to software. To use the default version of NWChem, type:

% module load nwchem

To see where the nwchem executables reside (the bin directory) and what environment variables it defines:

% module show nwchem

For example, on Cori:

% module avail nwchem
--------------------------- /usr/common/usg/Modules/modulefiles ------------------------
nwchem/6.6     nwchem/6.8(default)         nwchem-knl/6.8(default)

Then nwchem modules are for Cori Haswell, and nwchem-knl is for Cori KNL (Note the binaries built for Haswell run Cori KNL, but not vise versa). To see what environment variables the default nwchem module, nwchem/6.5, defines, do 

 

% module show nwchem
-------------------------------------------------------------------
/usr/common/software/modulefiles/nwchem/6.8:

module-whatis NWChem is a parallel computational chemistry package
setenv NWCHEM_DIR /global/common/cori/software/nwchem/6.8/hsw
setenv NWCHEM_BUILD_DIR /global/common/cori/software/nwchem/6.8/../build/nwchem-6.8
setenv NWCHEM_BASIS_LIBRARY /global/common/cori/software/nwchem/6.8/hsw/data/libraries/
prepend-path PATH /global/common/cori/software/nwchem/6.8/hsw/contrib
prepend-path PATH /global/common/cori/software/nwchem/6.8/hsw/bin
-------------------------------------------------------------------

How to Use NWChem on Cori

There are two ways of running NWChem, submitting a batch job, or running interactively in an interactive batch session. Here are sample batch job scripts.

To run NWChem with batch jobs

Sample job script for Cori Haswell:

#!/bin/bash -l
#SBATCH -J test_nwchem
#SBATCH -q debug
#SBATCH -N 2
#SBATCH -C haswell
#SBATCH -t 30:00
#SBATCH -o test_nwchem.o%j

module load nwchem

#for Cori which has 32 cores per node:
srun -n 64 nwchem test1.nw >& test1.out

Sample job script for Cori KNL:

#!/bin/bash -l
#SBATCH -J test_nwchem
#SBATCH -q debug
#SBATCH -N 2
#SBATCH -C knl
#SBATCH -t 30:00
#SBATCH -o test_nwchem.o%j

module load nwchem-knl

export OMP_PROC_BIND=true
export OMP_PLACES=threads
export OMP_NUM_THREADS=4

#for Cori KNL use 64 cores out of 68 available per node:
srun -n 32 -c16 --cpu_bind=cores nwchem test1.nw >& test1.out

Then submit the batch job script (assume the above job script name is test_nwchem.slurm) using the sbatch command, 

% sbatch test_nwchem.slurm

2) To run NWChem interactively

Use the salloc command to start a batch session, e.g., requesting 2 nodes:

On Cori Haswell:

%  salloc -N 2 -C haswell -q interactive -t 2:00:00    # requesting two Cori Haswell nodes 

Then when a new batch session prompts, issue the following commands:

module load nwchem

#for Cori which has 32 cores per node:
srun -n 64 nwchem test1.nw >& test1.out

On Cori KNL:

%  salloc -N 2 -C knl -q interactive -t 2:00:00    # requesting two Cori KNL nodes 

when a new batch session prompts, issue the following commands:

module load nwchem-knl

export OMP_PROC_BIND=true
export OMP_PLACES=threads
export OMP_NUM_THREADS=4

#for Cori KNL use 64 cores out of 68 available per node:
srun -n 32 -c16 --cpu_bind=cores nwchem test1.nw

Note that on Cori the use of the interactive QOS is recommended. Your job either gets the requested number of nodes or gets canceled in about 5 minutes. With the interactive QOS you can use up to 64 nodes for 4 hours. Each repo can use up to 64 nodes at any given time. 

Documentation

NWChem was developed by the High-Performance Computational Chemistry group in the Environmental Molecular Sciences Laboratory (EMSL) at the Pacific Northwest National Laboratory (PNNL).

NWChem User Manual

Availability

PackagePlatformCategoryVersionModuleInstall DateDate Made Default
NWChem cori applications/ chemistry 6.6 nwchem/6.6 2016-06-29
NWChem cori applications/ chemistry 6.8 nwchem/6.8 2018-12-28 2019-03-11
NWChem cori applications/ chemistry 6.8 nwchem-knl/6.8 2018-12-28 2018-12-28