NERSCPowering Scientific Discovery Since 1974

Batch Script Examples

My First Script

This is a simple example that you can use to make sure that your settings are correct before submitting more complicated jobs.

First, copy the contents of hello.sh into a file.

genepool% cat hello.sh
#!/bin/bash
sleep 120
echo "Hello World"

Then submit your job with the qsub command

genepool% qsub hello.sh

 Monitor your job with the qstat command:

genepool% qstat -u <username>

You can also get more detailed information about your job using:

genepool% qstat -j <jobid>

The job id can be found using the qstat -u command.

Basic Batch Script

Here is an example of a basic script that specifies the working directory, the shell and the queue.  The #$ must be used to specify the grid engine options used by qsub.  Details are provided for each of the options in the script.

#!/bin/bash
# Set SGE options:
## ensure a specific environment variable is set properly
#$ -v env_var=<value>
## run the job in the current working directory (where qsub is called)
#$ -cwd
## specify an email address
#$ -M uname@lbl.gov
## specify when to send the email when job is (a)borted, (b)egins or (e)nds normally
#$ -m abe
## Specify a run time of 11 hours (or 39600 seconds)
#$ -l h_rt=11:00:00
## Specify 12G of memory for the job
#$ -l ram.c=12G

## Your job info goes here
./myprogram inputs

Then the above is submitted using:

% qsub myscript

Whole-node Scheduling

If you would like to run on the new hardware, you need to specify the exclusive.c complex in your script.  This will give you access to nodes with 120G of memory and 16 physical cores.  This means that you can run with up to 16 threads or 16 MPI tasks.  

#!/bin/bash

#
# Set SGE options:
#
## run the job in the current working directory (where qsub is called)
#$ -cwd
## specify an email address
#$ -M uname@lbl.gov
## specify when to send the email when job is (a)borted, (b)egins, or (e)nds
#$ -m abe
## Request a whole compute node, the new hardware is all scheduled this way
#$ -l exclusive.c
## specify a 12 hour runtime
#$ -l h_rt=12:00:00
## specify the memory used, up to 120G of memory with ram.c
#$ -l ram.c=120G

# Your job info goes here
./myprogram inputs

Large Memory, Long Walltime Example

If your job requires more than the default 5GB of memory, you can do the following:

#!/bin/bash
#
# Set SGE options:
#
## run the job in the current working directory (where qsub is called)
#$ -cwd
## specify an email address
#$ -M uname@lbl.gov
## specify when to send the email when job is (a)borted, (b)egins, or (e)nds
#$ -m abe
## specify a 24 hour runtime
#$ -l h_rt=24:00:00
## specify the memory used, with ram.c
#$ -l ram.c=100G
## Your job info goes here
./myprogram inputs

High Priority Queue

If you want to test your pipeline on a small problem before submitting a job that requires more resources, use the high priority queue.  Each users is allowed to use 8 slots in the high priority queue.

#!/bin/bash
#
# == Set SGE options:
#
## run the job in the current working directory (where qsub is called)
#$ -cwd
## submit the job to the high priority queue
#$ -l high.c
## specify an email address
#$ -M uname@lbl.gov
## when to send the email when job is (a)borted, (b)egins, (e)nds
#$ -m abe
## specifying an 8 hour runtime
#$ -l h_rt=08:00:00
## specify 48G of memory
#$ -l ram.c=48G
#
# == Your job info goes here
./myprogram inputs