Using SCP/SFTP At NERSC
Overview
Secure Copy (SCP) and Secure FTP (SFTP) are used to securely transfer files between two hosts using the Secure Shell (SSH) protocol.
Suggested for smaller files (<~10GB)Availability
SCP and SFTP are available on all NERSC systems.
Requirements
To transfer files into/out of NERSC using SCP or SFTP, you need a SSH client:
- Linux/Unix/Cygwin: command ssh, scp or sftp
- Windows: Many GUI tools such as WinSCP
- MacOS: Many GUI tools such as Fugu
Usage
All example commands below are executed on your local machine, not the NERSC machine:The scp command
Get a file from Data Transfer Node
scp user_name@dtn01.nersc.gov:/remote/path/myfile.txt /local/path
Send a file to Data Transfer Node
scp /loca/path/myfile.txt user_name@dtn01.nersc.gov:/remote/path
Using tar+ssh
When you want to transfer many small files in a directory, you can consider using ssh piped with tar.
Send a directory to Data Transfer Node:
tar cz /loca/path/dirname | ssh user_name@dtn01.nersc.gov tar zxv -C /remote/path
Get a directory from Data Transfer Node:
ssh user_name@dtn01.nersc.gov tar cz /remote/path/dirname | tar zxv -C /local/path
Using the GUI SFTP Clients
Simply login to your favorite GUI SFTP Client with your NIM username and password. Then Drag and Drop files.
Pros and Cons
Pros
- Simple to use. No extra credential needed.
- SCP provides good status information when transferring
- Encrypted by default
Cons
- Does not provide turing options for WAN transfers
- Does not support parallel transfer
- Slower comparing to bbcp/GridFTP, only suggested for small files (or a bundle of small files)
