NERSCPowering Scientific Discovery Since 1974

Batch Jobs

Torque Keywords

The following table lists recommended and useful Torque keywords. For an expanded list of Torque job options and keywords see the qsub documentation but keep in mind that this is describes a generic Torque implementation and not all options and environment variables are relevant to or defined on Edison (for example, $PBS_NODEFILE is not defined).

Required Torque Options/Directives
OptionDefaultDescription
-l mppwidth=nodes*cores_per_node One node will be used. Used to allocate nodes to your job. The number of nodes you'll get is the value of mppwidth divided by the number of cores per node (24 for Edison unless you use HyperThreading), plus 1 if there is a remainder from the division
-l walltime=HH:MM:SS 00:30:00 Always specify the maximum wallclock time for your job.
-q queue debug Always specify your queue, which will usually be debug for testing and regular for production runs. See "Queues and Policies" in the left-hand menu.
Useful Torque Options/Directives
OptionDefaultDescription
-lmppnppn=MPI_tasks_per_node 24 (Edison) Use MPI_tasks_per_node tasks per node (Cray specific)
-lmppdepth=threads_per_MPI_task 1 Run threads_per_MPI_task threads per node; use for OpenMP (Cray specific)
-N job_name Job script name. Job Name: up to 15 printable, non-whitespace characters.
-A mXXX Your default repo Charge this job to the NERSC repository mXXX (necessary only if you have more than one NERSC repo)
-e filename <script_name>.e<job_id> Write STDERR to filename
-o filename <script_name>.o<job_id> Write STDOUT to filename
-j [eo|oe] Do not merge. Merge STDOUT and STDERR. If oe merge as standard output; if eo merge as standard error.
-m [a|b|e|n] a E-mail notification options:
a = send mail when job aborted by system
b = send mail when job begins
e = send mail when job ends
n = do not send mail
Options a,b,e may be combined.
-S shell Login shell Specify shell as the scripting language to use.
-V Do not import. Export the current environment variables into the batch job environment.

All options may be specified as either qsub command-line options (see below) or as directives in the batch script as #PBS options.  Note: if you use both, any command line options will override the corresponding options in the batch script.

Note:  The pvmem option is not implemented on the Cray version of Torque.  Jobs with a pvmem option will be queued, but they may never run.