Linux/rsync

From Omnia
< Linux
Revision as of 21:44, 30 November 2016 by Kenneth (talk | contribs) (→‎rsync server)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

rsync

rsync

Description

"rsync is a software application for Unix systems which synchronizes files and directories from one location to another while minimizing data transfer using delta encoding when appropriate. An important feature of rsync not found in most similar programs/protocols is that the mirroring takes place with only one transmission in each direction. rsync can copy or display directory contents and copy files, optionally using compression and recursion.

In daemon mode, rsync listens to the default TCP port of 873, serving files in the native rsync protocol or via a remote shell such as RSH or SSH. In the latter case, the rsync client executable must be installed on both the local and the remote host.

Released under the GNU General Public License, rsync is free software." (rsync - Wikipedia)

Notes

Usage

rsync (Use the --dry-run option for testing)
rsync -P rsync://rsync.server.com/path/to/file file	Only get diffs. Do multiple times for troublesome downloads
rsync --bwlimit=1000 fromfile tofile	Locally copy with rate limit. It's like nice for I/O
rsync -az -e ssh --delete ~/public_html/ remote.com:'~/public_html'	Mirror web site (using compression and encryption)
rsync -auz -e ssh remote:/dir/ . && rsync -auz -e ssh . remote:/dir/	Synchronize current directory with remote one

SSH

Standard format: 

rsync -avz -e ssh remoteuser@remotehost:/remote/dir /this/dir/

With ssh keys: 

rsync -avz -e "ssh -i /home/thisuser/cron/thishost-rsync-key" remoteuser@remotehost:/remote/dir /this/dir/

In reverse: 

rsync -avz -e "ssh -p 9999" linux/ trogdor@oeey.com:/www/linux-tested-stage/

References:

Compile

The --no-motd option is included with the latest source.

Source: 

http://samba.anu.edu.au/rsync/download.html

Compile: 

cd ~/src
# git clone git://git.samba.org/rsync.git
wget http://samba.anu.edu.au/ftp/rsync/rsync-3.0.7.tar.gz
tar -zvxf rsync-3.0.7.tar.gz
cd rsync-3.0.7
./configure --prefix=/opt/rsync
make
sudo make install

Copy root

# -x says not to cross fs boundaries
rsync -axv --progress / /newpu/ken/kesx/

Rsync Hardlink Copy Backup

SOURCE=[SOURCE_PATH]
MAX=10
rm -rf $MAX
for i in $( seq $MAX -1 2 ) ; do
  mv $(( i - 1 )) $i
done
cp -al 'current' 1  # this is the key here
rsync -avrt --delete $SOURCE 'current'

Readme

Full System Backup with rsync - ArchWiki - https://wiki.archlinux.org/index.php/Full_System_Backup_with_rsync

This rsync script allows creating a full backup copy across filesystems. It is setup so that the copy includes intact booting capabilities, optionally excluding selected files.
The approach has benefits over omitting system files by just copying personal data; if the system becomes corrupted in the main partition, overcoming the problem means booting into the backup as opposed to identifying and reinstalling affected programs.


How to Backup Linux? 15 rsync Command Examples - http://www.thegeekstuff.com/2010/09/rsync-command-examples/

Exclude

--exclude 'dir1'

rsync -avz --exclude 'dir1' source/ destination/
rsync -avz --exclude 'dir1' --exclude 'dir2' source/ destination/

References:

including only certain extensions

What you need is indeed a combination of --include and --exclude arguments. For example the following will replicate only mp3 files in a directory named source, or any of its subdirectories, to a directory named target: [1] 

rsync -a --include '*/' --include '*.mp3' --exclude '*' source/ target/

This will (1) include any directory or subdirectory, whatever its contents, (2) include all files with an mp3 extension, and (3) exclude all other files. Any number of other file types can be included by adding their one include pattern for each before the --exclude, e.g. --include '*.bmp'.

rsync server

Runs on TCP port 873 by default.

/etc/rsyncd.conf: 

motd file = /etc/rsyncd.motd
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock

# Default port: 873
#port = 873

[files]
   path = /home/public_rsync
   comment = Rsync Files Server
   read only = yes
   list = yes
   timeout = 300

Run server: 

rsync --daemon

List rsync shares: 

rsync rsync://localhost

Sync: 

rsync -avrt rsync://localhost/files .

Larger Sync Example: 

rsync --verbose  --progress --stats --compress --rsh=/usr/local/bin/ssh \
     --recursive --times --perms --links --delete \
     --exclude "*bak" --exclude "*~" \
     /www/* webserver:simple_path_name

xinetd

/etc/xinetd.d/rsync: 

# default: off
# description: The rsync server is a good addition to an ftp server, as it \
#       allows crc checksumming etc.
service rsync
{
        #disable = yes
        disable = no
        flags           = IPv6
        socket_type     = stream
        wait            = no
        user            = root
        server          = /usr/bin/rsync
        server_args     = --daemon
        log_on_failure  += USERID
}

service xinetd restart

yum install rsync xinetd

References:

keywords

Retrieved from "https://aznot.com/index.php?title=Linux/rsync&oldid=3882"