Using SCP/SFTP At NERSC
Secure Copy (SCP) and Secure FTP (SFTP) are used to securely transfer files between two hosts using the Secure Shell (SSH) protocol.Suggested for smaller files (<~10GB)
SCP and SFTP are available on all NERSC systems.
To transfer files into/out of NERSC using SCP or SFTP, you need a SSH client:
- Linux/Unix/Cygwin: command ssh, scp or sftp
- Windows: Many GUI tools such as WinSCP
- MacOS: Command-line tools ssh, scp or sftp. Many GUI tools such as FileZilla
UsageAll example commands below are executed on your local machine, not the NERSC machine:
The scp command
Get a file from Data Transfer Node
scp email@example.com:/remote/path/myfile.txt /local/path
Send a file to Data Transfer Node
scp /loca/path/myfile.txt firstname.lastname@example.org:/remote/path
When you want to transfer many small files in a directory, you can consider using ssh piped with tar.
Send a directory to Data Transfer Node:
tar cz /loca/path/dirname | ssh email@example.com tar zxv -C /remote/path
Get a directory from Data Transfer Node:
ssh firstname.lastname@example.org tar cz /remote/path/dirname | tar zxv -C /local/path
Using the GUI SFTP Clients
Simply login to your favorite GUI SFTP Client with your NIM username and password. Then Drag and Drop files.
Pros and Cons
- Simple to use. No extra credential needed.
- SCP provides good status information when transferring
- Encrypted by default
- Does not provide turing options for WAN transfers
- Does not support parallel transfer
- Slower comparing to bbcp/GridFTP, only suggested for small files (or a bundle of small files)