NERSCPowering Scientific Discovery Since 1974

Queues and Policies

Overview

Jobs must be submitted to a valid submit queue. Upon submission the job is routed to the appropriate execution queue. Users can not directly access the execution queues. 

Submit
Queue
Execution
Queue
NodesAvailable
Cores
Max
Wallclock
Relative
Priority
Run
Limit
Eligible
Limit
Queue
Charge
Factor
interactive interactive 1-8 1-64 30 mins 1 2 1 1.0
debug debug 1-32 1-256 30 mins 2 2 1 1.0




regular
reg_short 1-16 1-128 4 hrs 3 8 4 1.0
reg_small 1-16 1-128 48 hrs 3 6 3 1.0
reg_med 17-32 129-256 36 hrs 3 5 2 1.0
reg_big 33-64 257-512 24 hrs 3 3 1 1.0
reg_long 1-16 1-128 168 hrs 3 2 1 1.0
reg_xlong 1-4 1-32 504 hrs 3 2 1 1.0

reg_xlmem
xlmem_sm 1 8 72 hrs 3 2 1 1.0
xlmem_lg 1 32 72 hrs 3 2 1 1.0
low low 1-32 1-256 24 hrs 4 5 3 0.5
scavenger scavenger 1-32 1-256 6 hrs 5 2 1 0.0
serial serial 1 1 48 hrs 3 150 20 1.0

Policies

  • If a user reaches the run limit in a queue, their eligible limit becomes 0. That is, once at the run limit, no additional jobs will be considered for scheduling for that user in that queue.  Jobs that are not eligible for scheduling will remain in the "blocked" state until the user drops below the run limit.
  • The debug AND interactive queues are to be used for code development, testing, and debugging ONLY. Production runs are strictly prohibited from using the debug and/or interactive queues. User accounts are subject to suspension if they are determined to be using the debug and/or interactive queues for production computing. In particular, job "chaining" in the debug queue is not allowed. Chaining is defined as using a batch script to submit another batch script.
  • The scavenger queue is available only to users with a zero or negative balance in one of their repositories. This applies to both total repository balances as well as per-user balances.  If a user has multiple repositories, they should add the line "#PBS -A <repo>" to the jobscript in order to specify the repository against which a job is to be charged.
  • There are 8 nodes reserved from 5am to 6pm Pacific Time, Mon-Fri, for interactive and debug jobs.
  • There is a limit of 500 submitted jobs per user, per queue.
  • There is a limit of 165 running jobs per user across the entire system; note that the serial queue is exempt from this limit.
  • Jobs submitted to the reg_xlmem queue must specify a memory requirement.  The requested memory must be at least 16GB per task, and no more than 960GB for the entire job.   See Carver Memory Considerations for examples.
  • Jobs running in the reg_long and/or reg_xlong queue can make it difficult to schedule system maintenance activities.  While NERSC will make every effort to allow jobs to complete prior to maintenance events, there may be times when jobs in reg_long and reg_xlong must be terminated in order to perform critical maintenance.
  • Any job that has been in the queue for 14 days or more, and is in the "user hold" state, will be removed from the system. Note that this means:
    • Jobs may not be held for more than 14 days; and
    • Jobs older than 14 days may not be held.
  • See Carver Memory Considerations for additional information on requesting large-memory (48 GB) and extra-large-memory (1TB) nodes.
  • Jobs that ask for ppn greater than 8 will be queued but will never run on any queue except for reg_xlmem.  The  reg_xlmem queue will run jobs with values of ppn up to 32.
  • For all queues except for reg_xlmem, jobs whose pvmem*ppn is greater than the maximum available memory, 44GB, will be queued but will never run.