NERSC logo National Energy Research Scientific Computing Center
  A DOE Office of Science User Facility
  at Lawrence Berkeley National Laboratory
 
PackagePlatformVersionModule Docs
vasp bassi 4.6.28 vasp NERSCVendor
vasp bassi 4.6.31 vasp/4.6.31 NERSCVendor
vasp franklin 4.6.28.pkent vasp NERSCVendor
vasp franklin 4.6.31 vasp/4.6.31 NERSCVendor
vasp jacquard 4.6.28 vasp/4.6.28 NERSCVendor
vasp jacquard 4.6.31 vasp NERSCVendor
(*) Denotes limited support

VASP

VASP is a plane wave ab-initio code for quantum mechanical dynamics. It is highly scalable and shows very good parallel performance for a variety of chemical and materials science calculations. VASP is available to NERSC Users who already have an existing VASP license.

Setup

The modules package controls access to software. In order to access VASP at NERSC you must first verify your VASP license via email. When your request has been processed you will be able to access the vasp executables.

For instructions on how to obtain access to VASP at NERSC login to a NERSC machine and issue the command

% module what-is vasp

To use the default version of VASP, include the line:

% module load vasp

in your .profile.ext or .login.ext files, or type this command whenever you want to access VASP for a single session.

The version of VASP made available by this module uses MPI for parallelism. The plane wave transformations are done using the FFTW library.

VASP on Franklin

Currently we have VASP 4.6.28 (an optimized version by Paul Kent, module name vasp/4.6.28.pkent) and 4.6.31 (module name vasp/4.6.31) installed on franklin. The default version on franklin is 4.6.28.pkent, which should be significantly faster for large concurrency jobs. If you type

% module show  vasp
or
% module show  vasp/4.6.28.pkent
you can see where the VASP executables reside (the bin directory). There are several VASP executables available in the bin directory,
% ls -l /u0/z/zz217/USG/vasp/4.6.28.pkent/bin
total 49944
lrwxrwxrwx  1 zz217 usg       18 Oct  5 15:16 vasp -> vasp.franklin.prod
-rwxrwx---+ 1 zz217 usg 17216884 Sep 13 00:40 vasp.franklin.prod
-rwxrwx---+ 1 zz217 usg 16680736 Sep 13 00:36 vasp.franklin_gamma.prod
-rwxrwx---+ 1 zz217 usg 17237579 Sep 13 00:43 vasp.franklin_so.prod
These are "production" builds of the gamma point (executable name:vasp.franklin_gamma.prod), spin-orbit (vasp.franklin_so.prod), and general k-point but no spin-orbit version (vasp.franklin.prod) of the code, respectively. Note, the executable vasp is linked to vasp.franklin.prod, which is the general k-point collinear version. You need to choose an appropriate executable to run your job.
Similarly, if you type
% module show  vasp/4.6.31
you can see where the executables reside
% ls -l /u0/z/zz217/USG/vasp/4.6.31/bin
total 63456
-rwxr-x---+ 1 zz217 usg 16050634 Oct 17 10:21 vasp
-rwxr-x---+ 1 zz217 usg 16076194 Oct 17 10:34 vasp_ncl
-rwxrwx---+ 1 zz217 usg 16431860 Oct 17 12:18 vasp_ncl_neb
-rwxrwx---+ 1 zz217 usg 16410453 Oct 17 12:11 vasp_neb
drwxrws---+ 4 zz217 usg     4096 Aug  4 14:51 vtstscripts
These are the standard (collinear) vasp (executable name: vasp), non-collinear (vasp_ncl), Climbing-Image NEB (from G. Henkelman's group) enabled collinear (vasp_neb) and non-collinear vasp (vasp_ncl_neb). And the directory vtstscripts contains the shell and perl scripts for analysing results of VASP calculations (from G. Henkelman's group).
You can find the potential files in directory /usr/common/usg/vasp/4.6.31/.

Running on Franklin

There are two ways of running VASP on franklin, submitting a batch job, or running interactively in an interactive batch session.
1) Sample batch script to run VASP on franklin:

#PBS -N test_vasp 
#PBS -q debug
#PBS -l mppwidth=32
#PBS -l walltime=00:30:00
#PBS -j oe
#PBS -V 

cd $PBS_O_WORKDIR
module load vasp

aprun -n 32 vasp 

Then submit the job script using qsub command, eg., assume the job script name is test_vasp.pbs,
% qsub  test_vasp.pbs 

2) To request an interactive batch session, issue a command (eg., requesting 32 PEs):
% qsub  -I -V -q interactive -l mppwidth=32 
when a new batch session is returned to you, issue the following commands:
%  cd $PBS_O_WORKDIR 
%  module load vasp 
%  aprun -n 32 vasp 

Running on Bassi

After the module is loaded you can run vasp using

% vasp  -nodes #n -task_per_node #t 

There is a version of vasp built for non-collinear calculations. After the vasp module is loaded run the following command.

% vasp_ncl  -nodes #n -task_per_node #t 
Where #n and #t describe the total number of MPI tasks you've chosen. In a batch script the "-nodes" and "-tasks_per_node" need not be specified since they are given in the header to the batch script.
Sample batch script for Bassi:
#@ job_name        = vaspBatch
#@ output          = vasp.out
#@ error           = vasp.err
#@ job_type        = parallel
#@ environment     = COPY_ALL
#@ notification    = complete
#@ network.MPI     = sn_all,not_shared,us
#@ node_usage      = not_shared
#@ class           = regular
#
#
#@ node            = 1
#@ tasks_per_node  = 4
#@ wall_clock_limit= 01:00:00
#
#@ bulkxfer        = yes
#
#@ queue

module load vasp

vasp

Running on Jacquard

On Jacquard you need to load the acml module along with the vasp module.
% module load acml
% module load vasp
% mpirun -np 4 $pathToVasp/vasp

Or a sample batch script for Jacquard:
#PBS -S /usr/bin/tcsh
#PBS -l nodes=2:ppn=2,walltime=0:03:00
#PBS -N vasptest
#PBS -o job.out
#PBS -e job.err
#PBS -q debug
#PBS -V

module load acml
module load vasp
cd $PBS_O_WORKDIR
mpiexec -n 4 vasp

Performance and Scaling

  • We have found most VASP simulations fit well and run efficiently on 8-16 processors
  • In fact, small simulations show much worse performance running at higher concurrencies and are suspected of causing some system issues because the processors each have little work to do and spend a majority of time doing communication.
  • scaling of small vasp problem scaling of small vasp problem
  • Some problems requiring greater atom configurations benefit from running at higher concurrencies, however these problems quickly become load imbalanced running on 64 processors or more
  • scaling of larger vasp problem scaling of larger vasp problem

    Documentation and Help


    LBNL Home
    Page last modified: Fri, 22 Feb 2008 21:34:52 GMT
    Page URL: http://www.nersc.gov/nusers/resources/software/apps/materials_science/vasp/
    Web contact: webmaster@nersc.gov
    Computing questions: consult@nersc.gov

    Privacy and Security Notice
    DOE Office of Science