Batch Differences: NQE/NQS vs. LoadLeveler

This document provides information about important differences between Cray Network Queuing Environment (NQE) and IBM LoadLeveler batch systems.

• Commands
• Creating a batch job
• Example Job Command Files
• How to submit a batch job
• How to monitor a batch job
• How to cancel a batch job
• Additional LoadLeveler Features
• How to query machine status
• How to put hold on a job
• How to release a hold on a job
• A missing NQE Feature in LoadLeveler: checkpoint/restart

Commands

Creating a Batch Jobs

As with NQE, jobs are submitted to the LoadLeveler batch system via a job command file. A "command file " is a script file with LoadLeveler keywords (options) embedded as comments to give LoadLeveler information on how and where to run the job. LoadLeveler options start with "#@" whereas NQE options start with "#QSUB".

The LoadLeveler job command file consists of the following:

1. Embedded LoadLeveler keyword (option) statements (preceding any shell command statements).

   In order to submit a job for processing every LoadLeveler job command file must contain at least one "queue" statement that was preceded by the option statements relevant to the submitted job.

   For a comparison of the commonly-used NQE options with their LoadLeveler counterparts, see Analogous NQE options and LoadLeveler Keywords.

2. Shell command statements.

Example Job Command Files

• Example parallel job command files: NQE/ LoadLeveler
• Example serial job command files: NQE/ LoadLeveler

How to submit a batch job

After building a job command file, you can submit the job command file using the "llsubmit" command similar to NQE "cqsub" command.

Note: Unlike NQE, you can't specify LoadLeveler options on the command line.

You submit your job to a desired class (queue) similar to an NQE pipe queue. For differences in class terminology see The IBM SP Batch System.

For available classes, see Resource limits.

You can list information about the available classes (queues) by using the command "llclass" , which corresponds to NQE "qstat -b" command used to display batch queue information. See the "llclass" man page for details.

Priorities: Unlike with NQE on the Crays (where jobs can run at different priorities in each queue), you cannot specify a priority to LoadLeveler. The class will contain the name of the priority at which it will be charged.

When you submit a job, LoadLeveler assigns the job a "Job ID" (host.jobid). This is different than the NQE, in which a batch job is assigned both an NQE taskid and NQS request identifier.

How to monitor a batch job

After your job has been submitted to LoadLeveler, its status changes as it progresses through the batch system. You can check your job's status with the "llq/llqs" command similar to NQE "cqstatl/qstat" commands. See Table of LoadLeveler's most common job states as compared to NQE job substates.

How to cancel a batch job

To cancel one or more jobs from the LoadLeveler queue, use the command "llcancel", which corresponds to NQE "cqdel". See the "llcancel" man page for details.

Additional LoadLeveler Features

• How to query machine status

  You can display the status of machines in the LoadLeveler cluster (including load on individual nodes) with the command "llstatus". See the "llstatus" man page for details.

• How to put hold on a job

  Use the "llhold" command to put a hold on a queued job. See the "llhold" man page for details. Note: It does not affect running jobs.

• How to release a hold on a job

  Use the "llhold" command with the "-r" option to release a job on hold.