How usage is charged
Computer Usage Charging (Edison and Cori)
A parallel job is charged for exclusive use of each multi-core node allocated to the job. The base charge for each node used by the job is given in the following table.
|System||Node Architecture||Base Charge per Node Hour (NERSC Hours)||System Size (Nodes)||Cores per Node|
|Cori||Intel Xeon Phi (KNL)||96||9,303||68|
|Cori||Intel Xeon (Haswell)||80||2,004||32|
|Edison||Intel Xeon (Ivy Bridge)||48||5,576||24|
Example: A parallel job that uses 10 Cori Haswell nodes that runs for 5 hours accrues 80 x 10 x 5 = 4,000 NERSC Hours.
NERSC Hours are designed to allow a common currency to be used across different NERSC systems and architectures. Historically, 1 NERSC hour is approximately equivalent to the charge for 1 core hour on the retired Hopper system.
Modifications to the Base Charge
The base charges shown above are modified in the following circumstances.
|Premium Queue Priority||Cori and Edison||2 x base charge|
|Big Job Discount||Cori KNL jobs that use >=1,204 nodes||0.8 x base charge|
|Big Job Discount*||Edison jobs that use >683 nodes||0.8 x base charge|
|Shared Node||Cori Haswell nodes||2.5 NERSC hours per core hour|
|Scavenger||Cori and Edison||No charge ("free") once a repository allocation is exhausted|
* Through July 31, 2017
See the "Running Jobs" pages for each system for instructions on how to request low, premium, and shared. Big jobs are automatically detected and charged appropriately. Scavenger is automatically applied to jobs associated with repositories that have exhausted their allocation.
Charging your batch job to a specific project
If you do not specify which project account (called repository or repo) to charge, your charges will normally accrue to your default repository, unless you are out of time in that repository. To charge a batch job to a specific repository use the SLURM keyword:
#SBATCH -A repo_name
When jobs require KNL nodes to be rebooted, during the reboot the nodes are unavailable for other jobs, even though the job requesting reboot has not started actually running, therefore time spent rebooting is included in the time charged for the job.
HPSS Charging (Storage)
HPSS charging is based on allocations of Storage Resource Units (SRUs) which are awarded into accounts called HPSS repos. If a login name belongs to only one HPSS repo all of its usage is charged to that repo. If a login name belongs to multiple HPSS repos its daily charge is apportioned among the repos using the project percents for that login name. Default project percents are assigned by NIM based on the size of each repo's storage allocation.
Users can see their project percents on the Account Usage Summary page in NIM. This is the page that is displayed when you first login to NIM (or select My Account Usage from the NIM main menu). Look at the column Proj % in the HPSS repo display area.
To change your project percents, select Change SRU Proj Pct from the Actions pull-down list in the NIM main menu.
For more detailed information about HPSS charging see HPSS charging.
Running out of Allocation
Accounting information is sent from the computational machines and HPSS to NIM once daily (in the early morning, Pacific Time). At this time actions are taken if a repo or user balance is negative.
If a repo runs out of time (or Storage Resource Units) all login names which are not associated with another active repository are restricted:
- On computational machines restricted users are able to log in, but cannot submit batch jobs, or run parallel jobs, except to the "scavenger" partition.
- For HPSS restricted users are able to read data from HPSS and delete files but cannot write any data to HPSS.
Login names that are associated with more than one repo (for a given resource -- MPP or HPSS) are checked to see if the user has a positive balance in any of her or his repos (for that resource). If he or she does have a positive balance in some repo (for that resource) s/he will not be restricted and the following will happen:
- On computational machines the user will not be able to charge to the restricted repo. If the restricted repo had been the user's default repo the user will need to change their default repo through the NIM (Guide) or MyNersc interfaces, or specify a different repo with sufficient allocation when submitting a job.
- For HPSS, repos that are negative continue to incur SRU charges every day for each member that has HPSS files. This is because there is a daily charge for files stored within HPSS. Also, project percents are not adjusted when a repo goes negative. See HPSS Charging.
Likewise, when a user goes over her/his individual user quota in a given repo, that user is restricted if s/he has no other repo to charge to. A PI or Project Manager can change the user's quota.
Usage for MPP and HPSS resources are collected and displayed by the NERSC Information Management system, NIM. The two most commonly used usage reports in NIM are:
- My Account Usage: This is the default display for most users when they first log into NIM. It shows your personal usage by repository.
- Repository Usage: To see repository usage, type the repo name into the Repository box in the NIM main menu, then click Go.
There is also an inquiry-only command called getnim available on the computational systems. Users can use getnim interactively and in scripts to get their account balances.
The Dashboard in MyNERSC also shows individual user's repo balance and user balance.