Systems
HPSS Mass Storage
Accessing HPSS - HSI
HSI is a flexible and powerful interface utility to HPSS. The HSI commands are similar to those in ftp and pftp (e.g., put and mput) and UNIX (e.g., mv, mkdir, rm, cp, cd). HSI also has commands similar to those in CFS. HSI can be used both interactively or in batch scripts.
A related utility, HTAR, is useful for archiving multiple files to HPSS without using the intermediate local file storage that would be needed if one first used the tar utility followed by HSI.
There are man pages for HSI on production NERSC computers.
For documentation on HSI (version 3.4+) see the HSI 3.4 documentation.Authentication
Authentication on the HPSS systems is accomplished by using your NIM username and a HPSS authentication token, or by placing the NIM username and HPSS authentication token in a .netrc file in your HOME directory. See the HPSS Passwords page for details.
Connecting with HSI
To connect to the main user system (archive):
% hsi -h archive.nersc.gov
NERSC's other HPSS system - the original HPSS system at NERSC - is named "hpss". (It seemed like a good idea at the time, but now causes confusion. It is known internally as "regent"). This system is now used for backups, and does not offer the same capacity and performance as "archive." However, it does contain some older user data. To connect to it, use the command:
% hsi -h hpss.nersc.gov
Starting and Using HSI
HSI can accept input several different ways; some examples:
| From a command line: | hsi |
| Single-line execution: | hsi "mkdir foo; cd foo; put data_file" |
| From a command file: | hsi "in command_file" |
HSI can also read from standard input and write to standard output using pipes.
For "get" and "put" operations, HSI uses a special syntax to identify and separate the local and HPSS file names:
- The local file name is always on the left, and the HPSS file name is always on the right.
- A ":" (colon character) always separates the local pathname from the HPSS pathname, and the colon character must be surrounded by whitespace.
Examples:
% put local_file : hpss_file % get local_file : hpss_file
Recursive operations are allowed for the following commands:
cget, chgrp, chmod, chown, cput
delete, get, ls, mdelete, mget, mput
put, rm, stage, touch
Wildcards are supported.
Frequently Used Commands
HSI's command set is rich, and will look familiar to users of UNIX, FTP, and other storage utilities. A small set of commands will satisfy most user storage needs.
Short List of HSI Commands by Function
HPSS File and Directory Commands
| Command | Function |
|---|---|
| cd | Change current directory |
| get, mget | Copy one or more HPSS-resident files to local files |
| cp | Copy a file within HPSS |
| rm mdelete | Remove one or more files from HPSS |
| ls | List a directory |
| put, mput | Copy one or more local files to HPSS |
| pwd | Print current directory |
| mv | Rename an HPSS file |
| mkdir | Create an HPSS directory |
| rmdir | Delete an HPSS directory |
Local File and Directory Commands
| Command | Function |
|---|---|
| lcd | Change local directory |
| lls | List local directory |
| lmkdir | Make a local directory |
| lpwd | Print current local directory |
| command | Issue shell command |
File Administrative Information
| Command | Function |
|---|---|
| chmod | Change permissions of file or directory |
Miscellaneous HSI commands
| Command | Function |
|---|---|
| help | Display help information |
| quit, exit, end | Terminate HSI |
| in | Read commands from a local file |
| out | Write HSI output to a local file |
| log | Write all HSI commands and responses to a local log file |
| prompt | Toggles HSI prompting for mget, mput, and mdelete |
