Transferring files to Compute servers

All Compute servers share the same home directories. So any data you copy into these home directories will be available on all these systems.

Secure Copy (scp)

The most common way of transfering files is using scp, which is based on the SSH protocol. It uses the same authentication as SSH and provides the same security. The syntax for using the scp command is similar to the cp command.

scp [SRC] [DEST]

Remote pathnames are specified with the format

USER@HOST:PATH

USER is the user account on the remote system. If omitted it defaults to the environment variable $USER. PATH defaults to the home directory of the remote user if omitted. For example,

scp mycode.c ACCESSNET@compute.hpc.temple.edu:workdir

Copies the file mycode.c into the subdirectory workdir of the home directory of the user ACCESSNET. Passing the -r option to scp allows directories to be recursively copied. SCP also allows transfers between two remote hosts.


Remote Sync (rsync)

rsync is a utility for efficiently transferring and synchronizing files across computer systems, by checking the timestamp and size of files. The rsync algorithm is a type of delta encoding, and is used for minimizing network usage. It uses the same authentication as SSH and provides the same security.

Similar to scp, rsync requires the specification of a source and of a destination; either of them may be remote, but not both:

rsync [OPTION] … SRC … [USER@]HOST:DEST
rsync [OPTION] … [USER@]HOST:SRC [DEST]

USER is the user account on the remote system. If omitted it defaults to the environment variable $USER. SRC and DEST are the source and destination paths. These paths are defaults to the home directory in the case of remote location if omitted. For example,

rsync -r myFolder ACCESSNET@compute.hpc.temple.edu:workdir

Copies the folder myFolder into the subdirectory workdir of the home directory of the user ACCESSNET. Passing the -r option to rsync allows directories to be recursively copied. Besides the -r switch, there many other options that can be explored. For instance, here.