rsync

Everyone backs up their data. The problem I ran into is that I have 2TB’s of data to backup, and I couldn’t find a cloud based solution that uploaded my data fast enough. So, I went to plan B. As you can tell by the title of this blog, I am going to use rsync.

rsync is a software application and network protocol for Unix-like systems with ports to Windows that synchronizes files and directories from one location to another while minimizing data transfer by using delta encoding when appropriate. -Wiki-

I currently live in an apartment (source) and want a backup at my parents house (destination). I first did a full backup from one drive to another drive locally. This way, I am not trying to send 2TB of data over the internet with the first backup. It would literally take FOREVER. Once the backup was complete, I took my backup drive to my parents house and plugged it in to my server at home.

My Requirements:

Encrypt the data via SSH
Exclude hidden files
Only backup changes since the last copy
If data is deleted from source, delete from destination

Basic Usage:

$ rsync [OPTIONS] <source> <destination>

My most used options:
-h, –human-readable
-v, –verbose # increase verbosity
-r, –recursive # recurse into directories (meaning, include all sub-folders)
-a, –archive archive mode
-z, –compress # compress file data during the transfer
–delete delete extraneous files from dest dirs
-e, –rsh=COMMAND specify the remote shell to use
–ignore-existing skip updating files that exist on receiver
-n, –dry-run show what would have been transferred
–existing only update files that already exist on receiver
–exclude=PATTERN exclude files matching PATTERN Example: –exclude ‘.DS_Store’ OR –exclude=”.*” (excludes all hidden files)
-u, –update skip files that are newer on the receiver
–partial keep partially transferred files
-S, –sparse handle sparse files efficiently

Local > Remote:

$ rsync -av -e ssh --delete --exclude='.*' <source> <user_name>@<domain>:/<destination> -dry-run

Example:

$ rsync -av -e ssh --delete --exclude='.*' /Users/Tommy/Desktop/Source/ Tommy@itsmetommy.com:/Users/Server/Destination/ -dry-run

Use a for loop to list the folders within an External drive and sync them to a remote destination.

$ for i in `ls /Volumes/MyBook`;do rsync -av -e --delete --exclude='.*' /Volumes/MyBookSource/$i/ Tommy@itsmetommy.com:/Volumes/MyBookDestination/$i/;done -dry-run

Note:
Remove –dry-run when ready (you can also use -n in place of –dry-run).

Options used in my above command:

-a, –archive archive mode.
-v, –verbose # increase verbosity
-e, –rsh=COMMAND specify the remote shell to use
–delete delete extraneous files from dest dirs
–exclude=PATTERN exclude files matching PATTERN
-n, –dry-run show what would have been transferred

Local > Local:

$ rsync -av --delete --exclude='.*' <source> <destination> -dry-run

Example:

$ rsync -av –delete –exclude=”.*” /Volumes/MyBookSource/Music/ /Volumes/MyBookDestination/Music/ -dry-run

Use a for loop to list the folders within an External drive and sync them to a local destination.

$ for i in `ls /Volumes/MyBookSource`;do rsync -av --delete --exclude='.*' /Volumes/MyBookSource/$i/ /Volumes/MyBookDestination/$i/;done -dry-run

Note:
Remove –dry-run when ready (you can also use -n in place of –dry-run).

You can also write a cronjob to do a backup automatically if you’d like. See my cronjob blog HERE.

I kept it simple and straight forward. Now you have no excuses as to why your data is not backed up.