rsync
Category: /knowledge /linuxTags: linux
Tools
- rsync – rsync is a mirroring tool
- unison – Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other.
- rdiff-backup
rsync
Resources:
- Easy Automated Snapshot-Style Backups with Linux and Rsync with shell/perl scripts, excellent article
- With SSH information
useful commands
rsync -avzW --delete --exclude-from=./doc/exclude-rules.txt ~/workspace ~/doc mis183:.
### other than default port ###
rsync -e "ssh -p 2022" -Cauvzb .vim 128.175.7.162:.
### transfer a directory ###
rsync -t backup mis183:.
### copy/update a single file ###
rsync -auvz t.txt mis183:backup
### copy from a source list ###
The file names that are read from the FILE are all relative to the source dir -- any leading slashes are removed and no ".." references are allowed to go higher than the source dir. For example, take this command:
rsync -a --files-from=/tmp/foo /usr remote:/backup
If /tmp/foo contains the string "bin" (or even "/bin"), the /usr/bin directory will be created as /backup/bin on the remote host. If it contains "bin/" (note the trailing slash), the immediate contents of the directory would also be sent (without needing to be explicitly mentioned in the file -- this began in version 2.6.4). In both cases, if the -r option was enabled, that dir's entire hierarchy would also be transferred (keep in mind that -r needs to be specified explicitly with --files-from, since it is not implied by -a). Also note that the effect of the (enabled by default) --relative option is to duplicate only the path info that is read from the file -- it does not force the duplication of the source-spec path (/usr in this case).
In addition, the --files-from file can be read from the remote host instead of the local host if you specify a "host:" in front of the file (the host must match one end of the transfer). As a short-cut, you can specify just a prefix of ":" to mean "use the remote end of the transfer". For example:
rsync -a --files-from=:/path/file-list src:/ /tmp/copy
This would copy all the files specified in the /path/file-list file that was located on the remote "src" host.
I did some alterations as I wanted to backup more than just
the home dir. The include file determines what is to be
backed up now.
The new rsync command (now uses in and excludes and "/" as src) :
$RSYNC \
-va --delete --delete-excluded \
--exclude-from="$EXCLUDES" \
--include-from="$INCLUDES" \
/ $SNAPSHOT_RW/home/daily.0 ;
Exclude file (the stuff we dont want) : #rsync script exclude file
**/.pan/messages/cache/
**/.phoenix/default/*/Cache/
**/.thumbnails/
**/Desktop/Trash/
Include (what dirs to be included): #rsync script include file
/home/
/home/**
/var/
/var/www/
/var/www/**
/etc/
/etc/**
- *
Note the "- *" for excluding everything except the dirs mentioned
in the include file.
Also note the "/var/" entry. To backup /var/www/* , you need to
include /var/
One example to add directory names without contents
Subject: rsync exclude file - directory name without directory contents
Date: Wed, 03 Dec 2003 22:28:37 GMT
Newsgroups: alt.os.linux,comp.os.linux.misc,comp.unix.admin
When using rsync to maintain incremental backups significant space can be
saved on the backup server by excluding directories that have non-critical
or transient data such as /tmp, /proc, /mnt, /cdrom, object and backup
files, and caches. The problem is getting rsync to include the directory
name but exclude its contents, a condition not clearly covered in rsync(1),
EXCLUDE PATTERNS.
I use a separate backup server and pull from LAN clients but any model
should work. To test, add '-n -vv' to your rsync command and grep for
exclude and include statements. Feedback: rsync at nsr500 dot net.
Remote client example:
rsync -nvva --stats --delete --exclude-from=<file> -e ssh \
<client>:/ /<rsync_dir>/<client>/ | egrep '^excl|^incl'
The following rsync exclude file works on my RH 7.3 (2.4.23) and RH 6.2
(2.2.25) based systems for rsync 2.5.4. You may have to edit for other
distributions, especially non-linux or non-standard file system layouts.
--- rsync exclude file start ---
### transient directories - contents
### tmp ###
+ tmp/
- **/tmp/**
+ /tmpfs/
- /tmpfs/**
### transient ###
+ /proc/
- /proc/**
+ /mnt/
- /mnt/**
+ /cdrom/
- /cdrom/**
### cache ###
# mozilla
+ Cache/
- **/Cache/**
# ccache.samba.org
+ ccache/
- **/ccache/**
+ .ccache/
- **/.ccache/**
# others
+ cache/
- **/cache/**
+ xover-cache/
- **/xover-cache/**
### obj ###
# kernel build
- usr/src/**.o
# special library .o (may be RH specific?)
+ usr/*/lib/**.o
+ usr/lib/**.o
# all others
- *.o
### backup ###
- *~
--- rsync exclude file end ---
unison
Homepage: http://www.cis.upenn.edu/~bcpierce/unison/
rdiff-backup
Homepage: http://rdiff-backup.nongnu.org/