NERSCPowering Scientific Discovery Since 1974

HPSS Charging

Starting with the 2018 allocation year, HPSS archive storage will be managed by giving each group a space quota. Storage management and the quota system are intended to encourage efficient usage by the user community. The intention of the archive is to store valuable and unique data for long term retention.

HPSS quotas are reported and managed through the NERSC Information Management (NIM) system.  If a user has reached their quota in all of their HPSS repositories that user will be restricted so that they can no longer write data to HPSS (although they will continue to be able to read data).

Users can check their HPSS quotas with the hpssquota command on Cori or Edison. You view quotas on a user level:

cori03> hpssquota -u usgtest
HPSS Usage for User usgtest
REPO                          STORED [GB]      REPO QUOTA [GB]     PERCENT USED [%]
-----------------------------------------------------------------------------------
nstaff                             144.25              49500.0                  0.3
matcomp                              10.0                950.0                  1.1
-----------------------------------------------------------------------------------
TOTAL USAGE [GB]                   154.25

Here, "Stored" shows you how much data you have stored in HPSS. Data stored in HPSS could potentially be charged to any repo that you are a member of (see below for details). The "Repo Quota" shows you the maximum amount your PI has allocated for you to store data, and the "Percent Used" shows the percentage of the quota you've used.

Or on a repo level:

edison03> ./hpss_quota.py -r ntrain                               
HPSS Usage for Repo ntrain                                        

USER                          STORED [GB]           USER QUOTA [GB]          PERCENT USED [%]
---------------------------------------------------------------------------------------------
train1 100.00 500.0 20.0
train2                               0.35                      50.0                       0.1
train47                              0.12                     500.0                       0.0
train28                              0.09                     500.0                       0.0
---------------------------------------------------------------------------------------------

TOTAL USAGE [GB]         TOTAL QUOTA [GB]              PERCENT USED
100.56                              500.0                     20.11

"Stored" shows how much data each user has in HPSS that is charged to this repo. "User Quota" shows how much total space the PI has allocated for that user (by default this is 100%, PIs may want to adjust these for each user, see below for more info) and the "Percent Used" is the percentage of allocated quota each user has used. The totals at the bottom shows the total space and quota stored for the whole repo.

You can also check the HPSS quota for a repo by logging in to the NIM and clicking on their "Account Usage" tab.

Apportioning User Charges to Repositories: Project Percents

DOE's Office of Science awards an HPSS quota to each NERSC project every year. Users charge their HPSS space usage to the HPSS repos of which they are members. 

If a user belongs to only one HPSS repo all usage is charged to that repo. If a user belongs to multiple repos daily charges are apportioned among the repos using the project percents for that login name. Default project percents are assigned based on the size of each repo's storage allocation. Users (only the user, not the project managers) can change their project percents by selecting Change SRU Proj Pct (this is a historic name based on the old charging model) from the Actions pull-down list in the NIM main menu. Users should try to set project percents to reflect their actual use of HPSS for each of the projects of which they are a member. Note that this is quite different from the way that computational resources are charged.

  • On each computational system each job is charged to a specific repository. This is possible because the batch system has accounting hooks that handle charging to project accounts. The HPSS system has no notion of project accounting but only of user accounting. Users must say "after the fact" how to distribute their HPSS usage charges to the HPSS repos to which they belong. For a given project the MPP repository and the HPSS repository usually have the same name.

If users change their project percents this change will apply to all remaining days in the month but not to days prior to when the change was made.

If a user is added to a new repo or removed from an existing repo

If a user is added to a new repo or removed from an existing repo the project percents for that user are adjusted based on the size of the quotas of the repos to which the user currently belongs. However, if the user has previously changed the default project percents the relative ratios of the previously set project percents are respected.

As an example user u1 belongs to repos r1 and r2 and has changed the project percents from the default of 50% for each repo to 40% for r1 and 60% for r2:

LoginRepoAllocation (GBs)Project %
u1 r1 500 40
u1 r2 500 60

If u1 then becomes a new member of repo r3 which has a storage allocation of 1,000 GBs the project percents will be adjusted as follows (to preserve the old ratio of 40:60 between r1 and r2 while adding r3 which has the same SRU allocation as r1+r2):

LoginRepoAllocation (SRUs)Project %
u1 r1 500 20
u1 r2 500 30
u1 r3 1,000 50

User Quotas or Allowed Percents

The default User Quota or Allowed Percent for each user is 100% but Principal Investigators, PI Proxies, and Project Managers can change them for the users in their repo(s).

A user's HPSS allowed and used percentages as well as space balances are shown in NIM's Account Usage display:
% Used:

The percentage of the repo's space allocation that the login name has used
% Allowed:
The percentage of the repo's HPSS allocation that the login name is authorized to use (also known as the "user quota")
Balance:
The user's quota for this repo. A user's balance is computed by subtracting the usage within that repo from its "Allowed Percentage" of that repo. If the balance of the repository as a whole is less than the user's computed balance, than the lesser number (the repo's balance) is used instead. This user balance is shared with the other repo members.

User Statuses for HPSS

Within NIM the term User Status is used to display two sorts of statuses:  

Repository user statuses:  In the Account Usage Summary area NIM displays the User Status for each (login, repo) pair.  The Repository User Status (for the login name in that repo) is one of:

  • Active:  The user is a member of the repo and has a positive user balance in that repo.
  • Restricted:  The user is a member of the repo but has a negative user balance in that repo.
  • Limited:  The user is no longer a member of the repo but still has limited access to its resources.
  • Deleted:  The user has been remobed from this repo.
  • Admin Member:  The user is an administrative member (PI, PI Proxy or Project Manager) of the project who doesn't use this resource.

HPSS Repo User Statuses are also displayed in NIM's "project/repo" display area under the "HPSS Usage & Quotas" tab.

Machine user statuses (for computational resources or HPSS):   In the Login Info area under the Logins by Host tab NIM displays the status for each user on each machine the user has access to.  These Machine User Statuses have the following meaning for HPSS:

  • Active:  The user is in a normal active state with no restrictions.  The user can read and write data.
  • Restricted:  The user is restricted because it has no repo to charge to.  The user can read data from HPSS but cannot write to HPSS.
  • Disabled:  The user has been temporarily disabled.
  • Limited:  The user is restricted because the user is no longer a member of any active repository.  The user can read data from HPSS but cannot write to HPSS.  On HPSS a user remains limited for about one year prior to being deactivated.
  • Deactivated:  The user has been disabled and the user's HPSS files can be archived to the "crypt".  The user no longer has access to HPSS.
  • Crypt:  The user's files have been moved to the crypt and may be deleted at any time.
  • Deleted:  The user has been removed from HPSS.

What happens if a repo or user balance is negative?

Accounting information is sent from HPSS to NIM once daily (in the early morning, Pacific Time).  At this time actions are taken if a repo or user balance is negative.

If a repo runs out of SRUs all users in the repo are marked as restricted for that repo (see repository user statuses).

A user is "HPSS restricted" if all of the repos to which the user belongs are restricted (see machine user statuses).  HPSS restricted login names are able to read data from HPSS but cannot write any data to HPSS.

Likewise, when a user goes exceeds their individual "allowed percent" in a given repo the user is marked as restricted in that repo. The user is HPSS restricted only if the user status is restricted for each repo to which the user belongs.

Project percents are not adjusted when a repo goes negative or when a user exceeds their allowed percent.

HPSS Usage Reports

The following SRU Usage Reports are available in NIM on a daily, monthly, or yearly scale. You can access these reports from the NIM main menu. Select the Reports pull-down menu and choose either "Daily User HPSS", "Monthly User/Repo", or "Yearly User/Repo".