NERSCPowering Scientific Discovery Since 1974

Interactive Jobs

Interactive Jobs

The login nodes (genepool13 and genepool14) on Genepool should not be used for heavy interactive work.  These login nodes are shared amongst all Genepool users, so heavy CPU or memory usage will affect other Genepool users. Instead, you should run intensive interactive sessions via the scheduler with the "qlogin" command, or on a group-dedicated gpint server.

Scheduled Interactive Sessions

PLEASE NOTE: As of May 2016, qlogin sessions are, by default, run exclusively (meaning that all cores on a node will be given to the interactive session). Requests for interactive slotted sessions will now, typically, be un-schedulable (without clearing the JSV - see below). Interactive sessions are available to be run on any exclusive queue (normal_excl.q, high_excl.q, or long_excl.q), though users may experience issues with getting a session during peak usage hours (generally weekday afternoons). 

Use the qlogin command to run jobs interactively.  The example below shows how to request an interactive session on Genepool. 

genepool:$~> qlogin

This will put you directly onto the first available exclusive node, where you can then do interactive work.

Requesting a session on the less-frequently-used high_excl.q will often be more reliable, but depends heavily upon current cluster usage. Each user is permitted to use up to 2 sessions at a time in the high priority queue. 

genepool:$~> qlogin -q high_excl.q

 You can use qlogin to get on any compute node in the cluster, the high_excl.q option just routes you to the less utilized high-priority nodes. 

Suspending processes in a qlogin session

if you would like to suspend a process while running in a qlogin session (usually done with Ctrl-Z), you must do the following 

stty susp ^K 

Slotted interactive sessions

Slotted interactive sessions are NOT recommended. Although it is possible to request a slotted interactive session by clearing or avoiding Genepool's job submission verifier (JSV), other automated slotted jobs will be running on the compute node you qlogin into, and it would be possible, by accident or design, to interfere with those other jobs by consuming more memory or CPUs interactively than the qlogin initially requested. This can potentially cause harm to the JGI's pipelines, and so, by default, qlogin requests will go into exclusive queues. 

Group-specific Genepool Interactive Nodes (gpints)

Users can also run interactive work on the interactive/analysis nodes dedicated for various groups.  See the list of interactive resources available to your group.  One important aspect of using gpints is cooperation with your group members.  

Memory limits on gpint nodes

In order to prevent catastrophic system crashes due to overcommission of memory all genepool computational nodes (including login nodes and gpints) are configured to never overcommit memory.  This means that a 120G node will not be able to provide more than 120G of memory.  However, the "free" or "available" memory that you see in top output only accounts for memory not presently in use, it doesn't account for memory requested but not-yet claimed.  The only way at present to check right for the currently available memory is to look at the "Commited_AS" property in /proc/meminfo:

dmj@genepool12:~$ cat /proc/meminfo | grep -i commit
CommitLimit: 131668272 kB
Committed_AS: 16176344 kB

The information above indicates that the system believes it has a limit of 128GB (131,668,272 kB), and has allocated 15.4GB (16,176,344 kB) of memory.

WARNING: The default behavior of the Java Virtual Machine is to allocate 1/4 of the system memory.  On the login nodes we have set the limits very restrictively to prevent accidental overruns of memory.  You'll need to specify your maximum memory requirements carefully with java to prevent running the system too low on memory.  Please see