NERSCPowering Scientific Discovery Since 1974

gridFTP

Currently only the archive.nersc.gov system is capable of handling GridFTP transfers to HPSS.  It accomplishes this by using a special GSI enabled pftp server.  Data transfers are multi-threaded but are handled with a single FTP server.  Improvements are planned for the future.

The pftp server handling GridFTP transfers is garchive.nersc.gov.  GridFTP clients must authenticate/transfer to this server to send data to archive.nersc.gov.

There are numerous GridFTP clients available that can access the archive.nersc.gov system currently.  Several of these clients are already available on the NERSC computational systems.  The most common are globus-url-copy and uberftp (from NCSA). 

For details on using GridFTP see Grid Data Transfer and for details on connecting to HPSS via Globus Online, see the Globus Online page.

Currently, there is no explicit ordering by tape of file retrievals for Gripftp or Globus Online.  In retrieving a large data set with gridftp, we recommend that users see this page for instructions on how to best order files using HSI and then retrieve files using gridftp in tape order.

Calculating checksums via GridFTP is currently disabled and should not be used.

The major benefits of GridFTP clients over other HPSS clients are their ability to accept grid credentials that you may already have along with their multi-threaded transfer capability.  They are well known for making optimal utilization of available network bandwidth and performing well for WAN data transfers.  They can also do third-party transfers where you launch the transfer from a low-capability workstation for a transfer that occurs between two supercomputers.  Grid clients require additional software for this but can also launch and forget or retry on failure.