NERSCPowering Scientific Discovery Since 1974

Using SCP/SFTP At NERSC

Overview

Secure Copy (SCP) and Secure FTP (SFTP) are used to securely transfer files between two hosts using the Secure Shell (SSH) protocol.

Suggested for smaller files (<~10GB)

Availibility

SCP and SFTP are available on all NERSC systems.

Requirements

To transfer files into/out of NERSC using SCP or SFTP, you need a SSH client:

  • Linux/Unix/Cygwin: command ssh, scp or sftp
  • Windows: Many GUI tools such as WinSCP
  • MacOS: Many GUI tools such as Fugu

Usage

All example commands below are executed on your local machine, not the NERSC machine:

The scp command

Get a file from Data Transfer Node

scp user_name@dtn01.nersc.gov:/remote/path/myfile.txt /local/path

Send a file to Data Transfer Node

scp /loca/path/myfile.txt user_name@dtn01.nersc.gov:/remote/path

Using tar+ssh

When you want to transfer many small files in a directory, you can consider using ssh piped with tar.

Send a directory to Data Transfer Node:

tar cz /loca/path/dirname | ssh user_name@dtn01.nersc.gov tar zxv -C /remote/path

Get a directory from Data Transfer Node:

ssh user_name@dtn01.nersc.gov tar cz /remote/path/dirname | tar zxv -C /local/path

Using the GUI SFTP Clients

Simply login to your favorite GUI SFTP Client with your NIM username and password. Then Drag and Drop files.

Pros and Cons

Pros

  • Simple to use.  No extra credential needed.
  • SCP provides good status information when transferring
  • Encrypted by default

Cons

  • Does not provide turing options for WAN transfers
  • Does not support parallel transfer
  • Slower comparing to bbcp/GridFTP, only suggested for small files (or a bundle of small files)