NERSCPowering Scientific Discovery Since 1974

How usage is charged

Computer Usage Charging (Edison and Cori)

When a job runs on a Edison or Cori, charges accrue against one of the user's repository allocations.  The unit of accounting for these charges is the "NERSC Hour". 

Base Charge

A parallel job is charged for exclusive use of each multi-core node allocated to the job.  The base charge for each node used by the job is given in the following table.

SystemNode ArchitectureBase Charge per Node Hour (NERSC Hours)System Size (Nodes)Cores per Node
Cori Intel Xeon Phi (KNL) 96* 9,304 68
Cori Intel Xeon (Haswell) 80 1,988 32
Edison Intel Xeon (Ivy Bridge) 48 5,576 24

*Tentative value. Production computing value still to be determined. 

Example: A parallel job that uses 10 Cori Haswell nodes that runs for 5 hours accrues 80 x 10 x 5 = 4,000 NERSC Hours.

NERSC Hours are designed to allow a common currency to be used across different NERSC systems and architectures. Historically, 1 NERSC hour is approximately equivalent to the charge for 1 core hour on the retired Hopper system.

Modifications to the Base Charge

The base charges shown above are modified in the following circumstances.

 SystemCharge Modification
Premium Queue Priority Cori and Edison 2 x base charge
Low Queue Priority Cori and Edison 0.5 x base charge
Big Job Discount* Edison jobs that use >683 nodes 0.6 x base charge
Shared Node Cori 2.5 NERSC hours per core hour
Scavenger  Cori and Edison No charge ("free")
Cori Xeon Phi Pre-Production Cori Xeon Phi (KNL) nodes No charge ("free") until production computing begins on TBA

* Through June 30, 2017

See the "Running Jobs" pages for each system for instructions on how to request low, premium, and shared. Big jobs are automatically detected and charged appropriately. Scavenger is automatically applied to jobs associated with repositories that have exhausted their allocation.

Charging your batch job to a specific project

If you do not specify which project account (called repository or repo) to charge, your charges will normally accrue to your default repository, unless you are out of time in that repository.  To charge a batch job to a specific repository use the PBS keyword:

   #SBATCH -A repo_name

HPSS Charging (Storage)

HPSS charging is based on allocations of Storage Resource Units (SRUs) which are awarded into accounts called HPSS repos.  If a login name belongs to only one HPSS repo all of its usage is charged to that repo. If a login name belongs to multiple HPSS repos its daily charge is apportioned among the repos using the project percents for that login name.  Default project percents are assigned by NIM based on the size of each repo's storage allocation.

Users can see their project percents on the Account Usage Summary page in NIM.  This is the page that is displayed when you first login to NIM (or select My Account Usage from the NIM main menu).  Look at the column Proj % in the HPSS repo display area.

To change your project percents, select Change SRU Proj Pct from the Actions pull-down list in the NIM main menu.

For more detailed information about HPSS charging see HPSS charging.

Running out of Allocation

Accounting information is sent from the computational machines and HPSS to NIM once daily (in the early morning, Pacific Time).  At this time actions are taken if a repo or user balance is negative.

If a repo runs out of time (or Storage Resource Units) all login names which are not associated with another active repository are restricted:

  • On computational machines restricted users are able to log in, but cannot submit batch jobs, or run parallel jobs, except to the "scavenger" partition. 
  • For HPSS restricted users are able to read data from HPSS and delete files but cannot write any data to HPSS.

Login names that are associated with more than one repo (for a given resource -- MPP or HPSS) are checked to see if the user has a positive balance in any of her or his repos (for that resource).  If he or she does have a positive balance in some repo (for that resource) s/he will not be restricted and the following will happen:

  • On computational machines the user will not be able to charge to the restricted repo.  If the restricted repo had been the user's default repo, a new (temporary) default repo is assigned to the user (it can be any one of the user's remaining repos).
  • For HPSS, repos that are negative continue to incur SRU charges every day for each member that has HPSS files.  This is because there is a daily charge for files stored within HPSS. Also, project percents are not adjusted when a repo goes negative. See HPSS Charging.

Likewise, when a user goes over her/his individual user quota in a given repo, that user is restricted if s/he has no other repo to charge to. A PI or Project Manager can change the user's quota.

Usage Reports

Usage for MPP and HPSS resources are collected and displayed by the NERSC Information Management system, NIM.  The two most commonly used usage reports in NIM are:

  • My Account Usage: This is the default display for most users when they first log into NIM.   It shows your personal usage by repository.
  • Repository Usage: To see repository usage, type the repo name into the Repository box in the NIM main menu, then click Go.

There is also an inquiry-only command called getnim available on the computational systems.  Users can use getnim interactively and in scripts to get their account balances.

The Dashboard in MyNERSC also shows individual user's repo balance and user balance.