NERSCPowering Scientific Discovery Since 1974

Interactive Jobs

To run an interactive job on Edison's compute nodes you must request the number of nodes you want and have the batch system allocate resources from the pool of free nodes. The following command requests two nodes using the debug partition, and it requests a license to run on the $SCRATCH file system.  More details on requesting file system license can be found here

edison% salloc -N 2 -p debug -L SCRATCH

The -p flag specifies the name of the partition and the -N option specifies the number of nodes to allocate for your job. Since Edison has 24 cores per compute node, you can run up to 48 tasks in total on the two allocated nodes, 24 tasks/threads per node. If you use Hyperthreading, you may run up to 96 tasks/threads in total, 48 on each node. The debug partition has a 30 minute wall clock limit, so your interactive job will be terminated after 30 minutes from its start. If you do not need full 30 minutes, you can type the exit command to release the two nodes.

Assuming there are free nodes, and the salloc command returns your shell prompt, you will land on the head compute node, and will be in your work directory where the salloc command was executed. From the shell prompt, you can start your program on the compute nodes using the "srun" command, the parallel job launcher.

edison% srun -n 48 ./a.out

The srun command above launches 48 instances of the executable "a.out" on two Edison nodes, which will use all 48 cores. If your code is parallelized using MPI, then each instance is an MPI task and they can communicate via MPI routines. If a.out is "serial" then you will have 48 identical copies of the serial program running concurrently.

You don't have to launch 24 instances of your code on each node. By default, srun distributes job instances (MPI tasks) evenly across all the allocated nodes/sockets. The following sequence of commands will grab 6 Edison nodes and run 48 total instances (MPI tasks) of your code, 8 on each node (4 on each socket), and to request a license to use $SCRATCH file system. More details on requesting file system license can be found here

edison% salloc -N 6 -p debug -L SCRATCH
(wait for job to start ...)

edison% srun -n 48 ./a.out