NERSCPowering Scientific Discovery Since 1974

HTAR Configuration and Installation

HTAR is an archival utility similar to gnu-tar that allows for the archiving and extraction of local files into and out of HPSS. 

Configuration Instructions


This distribution has default configuration settings which will work for most environments.  If you want to use the default values (recommended) you can skip to the section labeled INSTALLATION INSTRUCTIONS.  In certain environments, for example if your installation is on a machine which has more than one network interface, you may want to change some of these default settings.  To help with this, an interactive Configure script is provided.  To use it do

$ ./Configure

prior to installing.  Configure will provide a description of the options you can change and allow you to set them as needed for your environment.  After collecting input Configure will write any changes to ./config_opts.  During the installation step config_opts is used to set basic configuration values.

Settable Options


HPSS Host Name: [defaults to the installation machine's hostname] This value can be assigned using either an IP address or a DNS name.

If HPSS Host Name is not set, then the network interface bound to the name returned by the 'hostname' command will be used for data transfers.  The interface specified by HPSS Host Name (or the one returned by the hostname command) must have a routable IP address over which the NERSC networks are reachable.

Use Port Range: [ default: OFF ] This option controls whether a port range is used for mover connections.  The default is to allow the operating system to assign the ports to use.  If setting this option to ON, a specific port range can be set with the "Mover Port Range" option.

Mover Port Range [default: 7500-7599] if Use Port Range is set to 'off', this option has no effect.  Otherwise, this option sets the range of network ports that are used for mover connections.  This allows a site to limit the number of ports over which they must allow incoming connections from NERSC networks.  To allow for parallel transfers, and for multiple clients to execute on the host at once, we recommend that a range of at least 100 ports be used.

Base Installation Directory [default: /usr/local] This is the installation root.  This value can be overridden during the installation step by setting the INSTALL_PREFIX variable.

Changes


Member file size: The individual member file size limit has increased from 8GB - 1 to 64GB - 1.  The limitation exists because the POSIX tar specification allocates 12 octal digits for storing the file size in the tar file record.

Authentication: HTAR now uses a NERSC specific authentication token.  The first time you connect to a NERSC HPSS, you will be prompted for your NIM password and an authentication token will be generated and stored in $HOME/.netrc.  After completing this step, you will be able to connect to HPSS without typing a password.  If needed, an alternative web based method for generating HPSS authentication tokens is available.  Please see HPSS Passwords for instructions.

Installation Instructions


This distribution uses the 'make' program to install - usage is:

Usage: make [INSTALL_PREFIX=installation_directory] install
            default: INSTALL_PREFIX=/usr/local

Examples:

1. To generation files for installation:
   $ make

2. To install into its default location (/usr/local) or the location set when running the Configure command do
   $ make install

   This will fail unless you have write permission for the installation directory.

3. To override the installation directory to be /opt/nersc
   $ make INSTALL_PREFIX=/opt/nersc install

   Note that you can also use INSTALL_PREFIX with the uninstall and clobber targets.

4. To clean up build area
   $ make clean

   This is especially useful if you run Configure multiple times to change options.

5. To uninstall
   $ make uninstall

6. To clean up build area, remove current installation, and attempt to remove old installations as well
   $ make clobber