HPSS Data Archive

• About The High Performance Storage System (HPSS) is a modern, flexible, performance-oriented mass storage system. It has been used at NERSC for archival storage since 1998.
• Getting Started You can access NERSC's HPSS systems from any machine that supports HSI, HTAR or ftp/pftp. You can also access the user system, archive.nersc.gov through, gridFTP clients. HPSS uses NIM and the NERSC LDAP server to create an "hpss token" for user authentication.
  • New Users Guide
  • HPSS Passwords
  • Accessing HPSS
  • HPSS Project Directories
• Storing and Retrieving Data
  • Clients
  • Software Downloads
  • Optimizing Performance
  • Using Sleepers to Manage Batch Access to HPSS
  • Mistakes to Avoid
• Advanced Usage Examples These pages demonstrate some useful techniques for using HSI and ftp/pftp, including their use in batch scripts.
  • Transferring Data from Batch Jobs
  • HSI Shell Script
  • PFTP Shell Script
  • HSI Tape Ordering Script
• HPSS Charging NERSC uses Storage Resource Units (SRUs) to help manage HPSS storage. SRUs are reported and managed via the NERSC Information Management (NIM) system. HPSS calculates SRUs for individual users and NIM distributes these user charges to the user's HPSS repos based on the user's HPSS project percents.
  • Storage Resource Unit (SRU) Formula Coefficients
  • SRU Calculator
• Storage Statistics Statistics about NERSC's storage systems are collected in order to plan for future needs and to monitor utilization.  The statistics come from HPSS collected statistics, HPSS accounting records and log files provided by the hsi and ftp…
  • Bandwidth and Transfer Activity
  • Storage Trends and Summaries
  • Storage by Scientific Discipline