NERSCPowering Scientific Discovery Since 1974

SC17 OpenMP Common Core Tutorial

OpenMP “Common Core,” A hands-on exploration

Note: a shortened (half day) version of this tutorial was given at IWOMP 2017

You can do the exercises either on an OpenMP enabled laptop or on the NERSC machines. To use NERSC, first logon to your training account. Although you can change your password through the interface at nim.nersc.gov, we recommend that you do not. The training account will enable you to access a reserved set of nodes for this tutorial. Substitute your training account as handed out for train171. After the actual tutorial, you can of course use a regular NERSC account without the reservation information.

ssh -X train171@cori.nersc.gov
(Type your password -- carefully please!)

Then recursively copy the tutorial exercises to your home directory. Use the period at the end of the command to specify the destination. You will see directories with a set of exercises in both C and Fortran. Pick your favorite.

% cp -r /project/projectdirs/training/SC17/omp-core/ .
% cd omp-core/OMP_Exercises

The tutorial exercises are intended to be run interactively on both the Cori systems (KNL and Haswell nodes). To use the Cori compute nodes you must request one node and have the batch system allocate resources from the pool of free nodes. Please grab/use only one node for your exercises. The following command requests one using either the debug partition or the special training partition.  You can compile on the login nodes, which are different from the compute nodes.

During the ACTUAL TUTORIAL use this box below. After the tutorial, use the debug queue (2nd box). The reservation will only work with training accounts, not your NERSC account. Run the make on the login nodes.

% make

To request a KNL node:
cori06% salloc -N 1 --reservation=sc17ompknl -p regalar -C knl,quad,cache -t 1:00:00
<Wait for your resource>

To request a Haswell node:
cori06% salloc -N 1 --reservation=sc17omphsw -p regular -C haswell -t 1:00:00
<Wait for your resource>
% make
cori06% salloc -N 1 --reservation=sc17omphsw -p special -C haswell -t 1:00:00
<Wait for your resource>

Skip this box below during the tutorial. You should already have a node from the reservation.

cori06% salloc -N 1 -p debug -C knl,quad,cache
<Wait for your resource>

or
cori06% salloc -N 1 --qos=interactive -C knl,quad,cache

To request a Haswell node:
cori06% salloc -N 1 -p debug -C haswell -t 1:00:00
<Wait for your resource>

or
cori06% salloc -N 1 --qos=interactive -C haswell -t 1:00:00
<Wait for your resource>

The -p flag specifies the name of the partition and the -N option specifies the number of nodes to allocate for your job. We are doing OpenMP only codes, so just grab one node (IMPORTANT not to hog nodes). The debug partition has a 30 minute wall clock limit, so your interactive job will be terminated after 30 minutes from its start. If you do not need full 30 minutes, you can type the exit command to release the node. --qos=interactive allows you to request up to 4 hrs.

Assuming there are free nodes, and the salloc command returns your shell prompt, you will land on the requested compute node, and will be in your work directory where the salloc command was executed. 

Now on a compute node:
nidxxxx% ./hello
nidxxxx% export OMP_NUM_THREADS=6
nidxxxx% ./pi_recur

Your training accounts should have the bash shell (which can be changed) however for this page instructions are given for that shell. 

The default exercises are set up to use the default Intel compiler on Cori.  To use gnu compiler on Cori, go to the C or Fortran subdirectory, 

% cd C  (or % cd Fortran)
% cp Make_def_files/cori_gnu.def make.def
% module swap PrgEnv-intel PrgEnv-gnu
% make

OMP_Common_Core_SC17 Slides