Resumable SFTP/SCP Transfers

Transmission errors inevitably occur during any online activity. This is specially true when tranferring large files. Murphy always prefers to poke his nose in at the worst possible time to disrupt an important file transfer.

Although HTTP and FTP transfers can be resumed without starting all over again, SFTP and SCP do not provide this capability out of the box. There are essentially two methods to obtain resumable SFTP and SCP transfers, but they are not 100% reliable. However, if you are stuck with an unreliable communication channel and a server supporting only SCP/SFTP, then it's the only ray of hope.

Method 1:
rsync supports resumable transfers across secure connections, and is the most easy to use solution. The downside is that both the server and the client must have rsync installed. If you're fortunate enough to have a server that supports rsync, the following command is all you will need.
rsync --partial --progress --rsh=ssh file_name user@host:server_path

If you use SCP on a daily basis, then it's worthwhile to add an alias to quickly start resumable transfers by editing ~/.bashrc and adding a line similar to:
alias scpr='rsync --partial --progress --rsh=ssh'
The invocation syntax will be similar to a normal scp command. For example:
scpr myfile.tar me@myremotehost.com:~/mydir/

Method 2:
cURL the multi-protocol URL grokker also provides resumable transfers with the -C flag.
curl -C -Tfile_name -u username sftp://myhost/mypath/

libcurl provided with my Fedora 9 setup is not built with libssh2 to enable SFTP and SCP. You can check whether your distribution has a libcurl that supports SCP/SFTP by running:
curl-config --protocols

I downloaded and compiled cURL from source to enable SCP and SFTP. However, I did not dare to replace the libcurl shipped with Fedora because there are over 85 programs depending on it. The downside is that my binary gets built by linking to the old libcurl which does not support SFTP and SCP ! However, I can run the curl static executable from the build directory with full SCP and SFTP support. Building a separate dynamic executable shouldn't be that difficult, but it requires wading through the Makefiles. Since I am not in a dire need for curl with SFTP and SCP support, I will defer this adventure to another time.