NERSCPowering Scientific Discovery Since 1974

SC18 OpenMP Common Core Tutorial

OpenMP “Common Core,” A hands-on exploration

You can do the exercises either on an OpenMP enabled laptop or on the NERSC machines. To use NERSC, first login to your training account. Although you can change your password through the interface at nim.nersc.gov, we recommend that you do not. The training account will enable you to access a reserved set of nodes for this tutorial. Substitute your training account as handed out for train171. After the actual tutorial, you can of course use a regular NERSC account without the reservation information.

ssh -X train171@cori.nersc.gov
(Type your password -- carefully please!)

Then recursively copy the tutorial exercises to your home directory. Use the period at the end of the command to specify the destination. You will see directories with a set of exercises in both C and Fortran. Pick your favorite.

% cd $SCRATCH
% cp -r /project/projectdirs/training/OpenMP_SC18 .
% cd OpenMP_SC18/C (or % cd OpenMP_SC18/Fortran)
% make

The default compiler is Intel.  To build an individual code instead of the provided Makefile, use compiler wrappers (ftn, cc, and CC) and the OpenMP compiler flag to build, such as:

% cc -qopenmp mycode.c

The tutorial exercises are intended to be run interactively on both the Cori systems (KNL and Haswell nodes). To use the Cori compute nodes you must request one node and have the batch system allocate resources from the pool of free nodes. Please grab/use only one node for your exercises. The following command requests one using either the interactive qos or the special training reservation.  You can compile on the login nodes, which are different from the compute nodes.

During the ACTUAL TUTORIAL use this box below. After the tutorial, use the interactive qos (2nd box). The reservation will only work with training accounts, not your NERSC account. Run the make on the login nodes.

To request a Haswell node:
cori06% salloc -N 1 --reservation=openmp_hsw -q regalar -C haswell -t 1:00:00
<Wait for your resource>

To request a KNL node:
cori06% salloc -N 1 --reservation=openmp_knl -q regular -C knl,quad,cache -t 1:00:00
<Wait for your resource>

After the tutorial day, use the batch interactive session as below to request a node:

cori06% salloc -N 1 --qos=interactive -C haswell -t 1:00:00
<Wait for your resource>

or
cori06% salloc -N 1 --qos=interactive -C knl,quad,cache -t 1:00:00
<Wait for your resource>

The -q flag specifies the name of the QOS and the -N option specifies the number of nodes to allocate for your job. We are doing OpenMP only codes, so just grab one node (IMPORTANT not to hog nodes).  --qos=interactive allows you to request up to 4 hrs.   If you do not need full 30 minutes, you can type the exit command to release the node.

Assuming there are free nodes, and the salloc command returns your shell prompt, you will land on the requested compute node, and will be in your work directory where the salloc command was executed. 

Now on a compute node:
nidxxxx% ./hello
nidxxxx% export OMP_NUM_THREADS=6
nidxxxx% ./pi_recur

Your training accounts should have the bash shell (which can be changed) however for this page instructions are given for that shell. 

The default exercises are set up to use the default Intel compiler on Cori.  To use gnu compiler on Cori, go to the C or Fortran subdirectory, 

% cd C  (or % cd Fortran)
% cp Make_def_files/cori_gnu.def make.def
% module swap PrgEnv-intel PrgEnv-gnu
% make

To build an individual code instead of the provided Makefile, use compiler wrappers (ftn, cc, and CC) and the OpenMP compiler flag to build, such as:

% module swap PrgEnv-intel PrgEnv-gnu
% cc -fopenmp mycode.c

OMP_Comm_Core_SC18 Slides 

OpenMP Common Core Reference Card