NERSCPowering Scientific Discovery Since 1974

How usage is charged

MPP Charging (Computational Systems)

When a job runs on a NERSC MPP system, such as Hopper, charges accrue against one of the user's repository allocations.  The unit of accounting for these charges is the "MPP Hour". 

A parallel job is charged for exclusive use of each multi-core node allocated to the job.  The MPP charge for such a job is calculated as the product of:

  1. the job's elapsed wall-clock time in hours
  2. the number of nodes allocated to the job (regardless of the number actually used)
  3. the number of physical cores available on each allocated node, when the node is used exclusively by the job
  4. a machine charge factor (MCF) based on typical performance of the machine relative to Hopper (MCF=1.0)
  5. a partition charge factor (PCF).  Available partitions are "debug", "regular", "shared", "realtime", etc. (not available on all systems)
  6. a QOS charge factor (QCF). Users can adjust job priority by using different QOS specifications.    A "premium" QOS job is scheduled for execution before an otherwise equivalent default "normal" QOS job.  A "low" QOS job has a lower priority for scheduling.  And a "scavenger" QOS job has lowest priority.

A "shared" job or a shared "realtime" job is charged for the time used by its number of physical cores used.  The MPP charge for such a job is calculated as the product of:

  1. the job's elapsed wall-clock time in hours
  2. the number of physical cores used on the node
  3. a machine charge factor (MCF) 
  4. a QOS charge factor (QCF)

Account information is available via the NIM web interface.  Charges are updated in NIM once a day (at about 3:00 AM Pacific time).

Batch Charging

The charge against your repo for a parallel batch job is:

(wallclock hours used) 
* (machine charge factor or MCF)
* (number of nodes allocated)
* (physical cores per node)
    * (QOS Charge Factor or QCF)

The charge against your repo for a "shared" or a shared "realtime" job is:

(wallclock hours used) 
* (machine charge factor or MCF)
* (number of physical cores used)
* (QOS Charge Factor or QCF)

The machine charge factor is:

  • 2.5 for Cori
  • 2.0 for Edison
  • 1.0 for Hopper (Hopper has retired, but it is used as our current machine base)

The QOS charge factor is:

  • 1.0 for normal QOS jobs   (this is the default if no --qos is specified)
  • 2.0 for premium QOS jobs
  • 0.5 for low QOS jobs
  • 0 for scavenger jobs

A 32-node regular partition, normal QOS job on Edison that begins at 12:00:00 and ends 8 hours later at 20:00:00, will have a charge in MPP hours of:

8 hours * 32 nodes * 24 physical cores/node * 2.0 MCF * 1.0 QCF = 6,144 MPP Hours.

Note that on Edison a discount is applied to jobs that run on a large number of processors.  See Batch Queues and Policies on Edison.

A shared partition job that used 8 physical cores (16 logical cores or CPUs as view in SLURM), premium QOS job on Cori that runs for 10 hrs will have a charge in MPP hours of:

10 hours * 8 physical cores * 2.5 MCF * 2.0 QCF = 400 MPP Hours

Charging your batch job to a specific project

If you do not specify which project account (called repository or repo) to charge, your charges will normally accrue to your default repository, unless you are out of time in that repository.  To charge a batch job to a specific repository use the PBS keyword:

   #SBATCH -A repo_name

HPSS Charging (Storage)

HPSS charging is based on allocations of Storage Resource Units (SRUs) which are awarded into accounts called HPSS repos.  If a login name belongs to only one HPSS repo all of its usage is charged to that repo. If a login name belongs to multiple HPSS repos its daily charge is apportioned among the repos using the project percents for that login name.  Default project percents are assigned by NIM based on the size of each repo's storage allocation.

Users can see their project percents on the Account Usage Summary page in NIM.  This is the page that is displayed when you first login to NIM (or select My Account Usage from the NIM main menu).  Look at the column Proj % in the HPSS repo display area.

To change your project percents, select Change SRU Proj Pct from the Actions pull-down list in the NIM main menu.

For more detailed information about HPSS charging see HPSS charging.

Running out of Allocation

Accounting information is sent from the computational machines and HPSS to NIM once daily (in the early morning, Pacific Time).  At this time actions are taken if a repo or user balance is negative.

If a repo runs out of time (or Storage Resource Units) all login names which are not associated with another active repository are restricted:

  • On computational machines restricted users are able to log in, but cannot submit batch jobs, or run parallel jobs, except to the "scavenger" partition. 
  • For HPSS restricted users are able to read data from HPSS and delete files but cannot write any data to HPSS.

Login names that are associated with more than one repo (for a given resource -- MPP or HPSS) are checked to see if the user has a positive balance in any of her or his repos (for that resource).  If he or she does have a positive balance in some repo (for that resource) s/he will not be restricted and the following will happen:

  • On computational machines the user will not be able to charge to the restricted repo.  If the restricted repo had been the user's default repo, a new (temporary) default repo is assigned to the user (it can be any one of the user's remaining repos).
  • For HPSS, repos that are negative continue to incur SRU charges every day for each member that has HPSS files.  This is because there is a daily charge for files stored within HPSS. Also, project percents are not adjusted when a repo goes negative. See HPSS Charging.

Likewise, when a user goes over her/his individual user quota in a given repo, that user is restricted if s/he has no other repo to charge to. A PI or Project Manager can change the user's quota.

Usage Reports

Usage for MPP and HPSS resources are collected and displayed by the NERSC Information Management system, NIM.  The two most commonly used usage reports in NIM are:

  • My Account Usage: This is the default display for most users when they first log into NIM.   It shows your personal usage by repository.
  • Repository Usage: To see repository usage, type the repo name into the Repository box in the NIM main menu, then click Go.

There is also an inquiry-only command called getnim available on the computational systems.  Users can use getnim interactively and in scripts to get their account balances.

The Dashboard in MyNERSC also shows individual user's repo balance and user balance.