Running on Bassi

Interactive jobs

Serial Jobs

You run interactive serial programs on the node you are currently logged onto by typing the executable file's name, e.g.:

bassi%  ./a.out

Processes run on the login nodes are subject to limits on certain resources such as memory. These limits are in place to make sure that the login nodes are available and responsive to all users logged in. If your program needs greater resources than provided by the interactive limits, you should run it in batch. There are both soft (default) and hard (maximum) limits. The limits are:

You query and change these limits with the limit command for csh and tcsh users and with the ulimit command for ksh, sh, and bash users.

Query the limits:

% limit 		(csh, tcsh: shows limits in effect)
% limit -h		(csh, tcsh: shows maximum values) 
% ulimit -a 		(bash, sh, ksh: shows limits in effect)
% limit -aH		(bash,sh, ksh: shows maximum values) 

% limit datasize 2097151 	(csh, tcsh)
% ulimit -d 2097152		(bash, sh, ksh) 

You cannot run interactive serial jobs on any of the "compute" nodes.

Parallel Jobs

Interactive parallel programs are executed by the Parallel Operating Environment (POE) software. POE can run the job only on the "compute" nodes. Parallel jobs do not run on the login nodes that are used for interactive terminal sessions and interactive serial jobs.

Interactive jobs are run through the LoadLeveler job scheduling software in the interactive class. Therefore interactive jobs are subject to class resource limits and policies associated with the interactive class.

To run a parallel job interactively:

1. compile your program with one of the "parallel" compiler invocations, e.g. mpxlf
2. use the poe command to invoke the Parallel Operating Environment.

NOTE: Use of the poe command is optional for programs compiled with one of the "parallel" compiler invocations. Those programs will execute just as if you had typed poe at the command line. In fact, there is no way to run an executable compiled with an "mp" compiler as a "serial" program.

Options can be passed to poe on the command line or by setting environment variables. Command line options override the environment variable settings.

Interactive jobs are charged to your default repository, unless you specify otherwise. See IBM Accounts & Charging.

Required POE flags

You should specify two of the following:

If you choose not to run with tasks per node set to the number of processors on the node, LoadLeveler will allocate tasks to the nodes in a balanced fashion.

There are a number of other available flags available. Two of the most useful flags are -retry N -retrycount M. These options specify an attempt to launch your parallel job should be made M times, with wait of N seconds between launch attempts. This is a good way of running an interactive job when the machine is busy.

Note that these are not the LoadLeveler keywords that are used in batch scripts, even though some of the names may be similar or even identical. Command-line arguments that poe does not recognize are passed to your program as arguments without warning or comment by poe.

See Operation and Use, Volume 1 Using the Parallel Operating Environment for POE command line flags and environment variables.

For example, to run a parallel job with N total tasks on M "compute" nodes, with 10 attempts to run the job, waiting 30 seconds between attempts, you would type:

bassi% poe ./a.out -procs N -nodes M -retry 30 -retrycount 10