I never remember how to do this, so for my own personal blog-memory:

rsync --partial --progress --rsh="ssh -p <remote ssh port>" <from file> <partially downloaded file>

• The --rsh parameter can contain all of your normal ssh command line-fu such as keys, options, ports etc. (Who runs public ssh on port 22 these days anyway?)
• The <from> and <to> follow the normal SCP format of <username>@<hostname>:<filename> or just <filename> for the local copy.