NERSCPowering Scientific Discovery Since 1974

Hands-On Exercises

The following commands are to be run from a Linux, UNIX, BSD, or Mac OSX command prompt. You can do the same functions from a GUI or third-party application, but the procedure will be application-dependent.

mymachine% - commands typed at your local machine prompt
hopper% - commands to be typed at the hopper command prompt

Connecting to NERSC

SSH to

mymachine% ssh -l <username> -Y -A

Look around on Hopper

hopper% ls
hopper% ls -a
hopper% myquota
hopper% module list
hopper% module avail
hopper% mkdir hopper
hopper% xtnodestat
hopper% emacs

If X Windows is configured properly, emacs will open in GUI mode; if not it will open in terminal mode (CONTROL-X CONTROL-C to exit)

Copy a file to NERSC from your local machine

mymachine% scp <filename> <username>

Copy training materials to your $SCRATCH directory

hopper% module load training
hopper% cp -r $EXAMPLES/NewUser $SCRATCH
hopper% cd $SCRATCH/NewUser

Explore the NERSC web site


Find the tutorial files you copied to $SCRATCH above.

hopper% cd $SCRATCH/NewUser
hopper% cd hello
hopper% make

You will see the commands used to compile and link the code. To see the benefit of the "ftn" wrapper, try the following to see all the command-line options you would need without it.

hopper% ftn -v hello.f90

Let's swap compiler environments. First check that PGI is the default compiler, then swap modules to use the GCC/GNU Fortran environment.

hopper% ftn -V
hopper% cc -V
hopper% module swap PrgEnv-pgi PrgEnv-gnu
hopper% ftn --version
hopper% cc --version
hopper% make clean
hopper% make

With that last make command, you've built the code using GNU instead of PGI. Note that even though the wrappers "ftn" and "cc" are the same, the command-line options are still specific to the underlying compiler.

Running Jobs

Try running some of the codes in the tutorial directories.

Submit a batch job

hopper% cd $SCRATCH/NewUser/flip
hopper% cat flip.pbs #Take a look at the script
hopper% qsub flip.pbs
hopper% qs -u <username>

The qs command will show your job's status. In the ST column, Q means waiting in the queue, R means that the job is running, and C means the job has completed. Once the job finishes, you will find output files in the directory you submitted from.

Try creating a batch script to run one of the other codes in the tutorial. Smaller, shorter job start sooner. Limit the size of your jobs to 96 CPU cores/MPI tasks so more people can run without waiting.

Run interactive parallel jobs

Depending on the system load and the number of people trying to run interactively, you may not be able to get an interactive session. If not, try again a little later.

hopper% cd $SCRATCH/NewUser/bcast
hopper% make
hopper% qsub -I -lmppwidth=24
hopper% cd $PBS_O_WORKDIR
hopper% aprun -n 24 ./bcast

You can poke around in any of the directories and run with a similar aprun command. The session stared by qsub reserved one Hopper node for you. So your aprun commands must not request more than 24 tasks with the -n option.

Try your own codes

If you have a code you'd like to run, copy it to Hopper and try to compile and run it. If your code need dynamic libraries, please see Shared and Dynamic Libraries first.  If you have trouble using Hopper, you can try