NERSCPowering Scientific Discovery Since 1974

GAMESS

Code Description

GAMESS: ab initio Molecular Quantum Chemistry


General Description

The benchmark code GAMESS (General Atomic and Molecular Electronic Structure System) performs various ab-initio quantum chemistry simulations. Several kinds of SCF wavefunctions can be computed with correlation, CI, Correlation corrections to these SCF wavefunctions include Configuration Interaction, second order Perturbation Theory, and Coupled-Cluster approaches, as well as the Density Functional Theory approximation. A variety of molecular properties, ranging from simple dipole moments to frequency dependent hyperpolarizabilities may be computed. Many basis sets are stored internally, together with effective core potentials, so all elements up to Radon may be included in molecules. [GAM] The two benchmarks used here test DFT energy and gradient, RHF energy, MP2 energy, and MP2 gradient.

Relation to DOD MOD TI-06

This version of GAMESS is the same as that in the DOD TI-06 (22 November 2004 R1) procurement [TI-06] except that ddi/shmem/ddishm.src and ddi/shmem/ddio3k.src were modified to support shmem targets for Cray and SGI. These modified files are allowed by TI-06. The medium and large test cases correspond to STD and LARGE test cases of TI-06. See Running and Reporting, below, for the differences in reporting results between Ti-06 and NERSC5.

Coding

The code consists of about 550,000 lines of Fortran90 code. It includes a communication layer called DDI (Distributed Data Interface) that makes use of highly optimized vendor libraries. BLAS routines are also used.

Authorship

GAMESS was developed and is maintained by the research group of Professor Mark Gordon of Iowa State University. Important contributors included M. Dupuis, D. Spangler, J. J. Wendoloski, S. Elbert, R. Kendall, and T. Windus.

Relationship to NERSC Workload

GAMESS is available and is used on three NERSC systems: Jacquard, Seaborg, and Bassi. The NERSC workload that uses it is funded by BES.

Parallelization

GAMESS has a somewhat complicated method of parallel operation. For an N-way parallel job, 2N MPI processes are required. Half of these processes perform data-handling only, while the other half do computation. The only time when this is not true is when GAMES is running within a single SMP enclosure, when using the Cray SHMEM library, or when using LAPI, e.g. the IBM SP. This method attempts to implement a load-balanced replicated-data message passing model.

Obtaining the Code

For NERSC-related procurements please visit the procurement site.

Due to licensing restrictions GAMESS source code cannot be downloaded from the NERSC site. To obtain the code visit the Gordon Group site at the Ames Lab.

You can obtain the NERSC-6 GAMESS benchmark input data files here (gzip tar file).  In the "tests" subdirectory you will find the nersc6-large.inp and nersc6-medium.inp files.

Building the Code

The following steps required to build the code.

  1. Go into the tools directory. Edit the file actvte.code, replace all strings *UNX with 4 blanks and save the file as actvte.f. There are additional vendor specific substitutions to be performed for Cray and NEC - please read the instructions at the top of actvte.code for additional information. Compile actvte.f with a Fortran compiler to produce the GAMESS source preprocessor "activator," actvte.x, e.g.
  2. f77 -o actvte.x -O actvte.f
  3. Return to the main GAMESS directory. Edit the files compall, comp, and lked, changing the values of the TARGET variable to one appropriate for your architecture. Some targets available include ibm64-sp, amd64, linux-ia64, and cray-x1; the complete list is in the file misc/readme.unix. In the same three files also change the chdir <directory> command to point to where you unpacked the GAMESS source code. Also correct the names and options of the compilers in the comp, and lked scripts. Note that compiler options for a given target architecture can be in many places in the script; e.g., some files are compiled with increased optimization, others with decreased optimization.
  4. GAMESS provides its own, "plain vanilla" Fortran source code for BLAS. If you have fast vendor provided BLAS routines available, remove the compilation of 'blas' from 'compall' (around line 74) and check that the optimized versions are linked correctly in lked. If you are compiling with long integers and you have a BLAS library that doesn't support these take a look at what happens with "blaswrap" in compall, comp, and lked. The "ibm64" target is an example of how this problem is fixed via blaswrap.
  5. Go to the ddi directory and edit the compddi script. Change line 17 to the correct architecture. The choices are the same as in step 2, above; common ones might include amd64, compaq-axp, cray-x1, ibm64-sp, or linux-ia64. Change line 34 to be the correct communications architecture; likely choices are "sockets," "mpi," "lapi," or "shmem." Change lines 45-46 to the appropriate values for number of CPUs and number of nodes in your system. Full instructions are in readme.ddi.
  6. Keep in mind that the compiler used in comp and compall is for single-processor compiles (we think) while the compiler used in compddi and in lked is probably for MPI compiles and is generally a wrapper script such as mpicc.
  7. Execute the compddi script to compile and build the ddi library. You can edit the compddi script to enable shmem or mpi for an new architecture but you should read the instructions in the ddi.readme file to determine the correct settings. The relocatable object files from this step go in a ddi/obj directory.
  8. After ddi/libddi.a is produced return to the main directory and execute the script compall to compile the GAMESS source code. After compall is finished, produce a GAMESS executable by executing lked. The executable is called "gamess.00.x" and it is in the same directory.


Build-Related Files in this distribution

A description of the directory structure and some important files is given in the following table.

Directory or File

Description

./


Top-level directory with README

./src.pmemd


Directory with top-level Makefile

./src.pmemd/pmemd


Directory: All source files, Makefile

./src.pmemd/Machines


Directory: Contains several Machine.<X> files with machine-specific environment variable definitions.

./src.pmemd/Machine


File: soft link to ./src.pmemd/Machines/.<X>

./src.pmemd/Machines/X


Directories for various machines containing system-dependent source code (sys_proto.f90 and erfcfun.c

exe


Directory: Executable is put here

Running the Code

The code is run by executing a shell script (provided) called rungms. Instructions:

  1. Edit rungms. Set the TARGET variable on line 37 to the correct value for your architecture. Set the SCR variable on line 38 to a directory where large temporary files can reside. Set the ERICFMT variable on line 89 to point to the ericfmt.dat file included in top-level directory of the distribution. Also make sure that the path to the GAMESS executable is correct; this will be in several places and depends on your OS. May have to change SMP_SIZE.
  2. Invoke the application with the following syntax (csh example):
rungms <size>.inp 00 <procs>  >&  <size>.log

where <size> is "small," "medium" or "large," and <procs> is the correct concurrency. Example:

rungms small.inp 00 4  >&  small.log            


Computational nodes employed in the benchmark must be fully-packed; that is, the number GAMESS "computational tasks" must be equal to the number of physical processors on the node. (If you use "generic" MPI-DDI then the number of MPI processes will be double the number of GAMESS "computational tasks."

Timing Issues


Storage Issues

Almost all arrays in GAMESS are allocated dynamically although some are held in Fortran COMMON.

Memory Required By The Sample Problems:

Small

To be determined GB

Medium

To be determined GB

Large

To be determined GB

XL

To be determined GB

The minumum memory configuration required to run the problems in each configuration must be reported (OS + buffers + code + data + ...).

Required Runs

There are three input decks supplied in the benchmark directory: small, medium, and large. Their intent is not to measure scalability; in fact, they compute different systems and do different kinds of computations.

The small case should be used for testing. The benchmark timings are required for the medium and large cases. Each case must be executed with a the concurrency shown in the following table.

Problem Size

Concurrency

small

4

medium

64

large

384

Verifying Results

The GAMESS benchmark directory contains a verify script that you should run on the output from a run. The syntax is

   verify <size> <your output file>

Succesful verification requires self checking.

Modification Record

This is GAMESS Release x

Record of Formal Questions and Answers

No entries as yet.

Bibliography

 [GAM] GAMESS Home Page http://www.msg.ameslab.gov/GAMESS/

[ TI-06] Technology Insertion-06 Documentation http://www.hpcmo.hpc.mil/Htdocs/TI/TI06/index.html