How to use Rsync and Grsync to securely copy files
Lets start with a simple case where we want to copy a single file from source to destination...
cd /tmp/ ls -lrt /home/downloads/SampleData.xlsx -rw-rw-r-- 1 root root 65801 Oct 31 2018 /home/downloads/SampleData.xlsx
Lets copy the above file /home/downloads/SampleData.xlsx to our current directory /tmp
rsync /home/downloads/SampleData.xlsx . pwd /tmp ls -lrt SampleData.xlsx -rw-r--r-- 1 root root 65801 Aug 22 07:24 SampleData.xlsx
Yes we got the file in the directory /tmp/ directory but the time stamp has changed.
To keep the original timestamp
rsync -t /home/downloads/SampleData.xlsx . ls -lrt SampleData.xlsx -rw-r--r-- 1 root root 65801 Oct 31 2018 SampleData.xlsx
Now we got the file and also preserved the timestamp
If the file is too big, use the -z compress switch, Rsync will compress the files before transfer
rsync -zt /home/downloads/SampleData.xlsx .
You can also use verbose mode to print information
rsync -ztv /home/downloads/SampleData.xlsx . sent 44 bytes received 12 bytes 112.00 bytes/sec total size is 65,801 speedup is 1,175.02
Lets copy a directory now. Following scripts directory has few files in it.
get_free_memory.sh get_ipaddr.sh prepare.sh
Lets try to copy the whole directory scripts with all the files in it
rsync -ztv /home/downloads/home/scripts . skipping directory scripts sent 16 bytes received 12 bytes 56.00 bytes/sec total size is 0 speedup is 0.00
As we see above the same didnt work. We need to add the switch -a (archive)
rsync -ztva /home/downloads/home/scripts/ . sending incremental file list ./ get_free_memory.sh get_ipaddr.sh prepare.sh sent 1,059 bytes received 76 bytes 2,270.00 bytes/sec total size is 1,766 speedup is 1.56
Seems like it has worked, let see if we got all the files and also the scripts directory
pwd /tmp ls -lrt scripts ls: cannot access scripts: No such file or directory ::
Seems like we didnt get the scripts directory, Lets see if we got the files...
ls -lrt get_free_memory.sh get_ipaddr.sh prepare.sh -rwxr-xr-x 1 root root 47 May 28 2017 [0m[01;32mget_free_memory.sh[0m -rwxr-xr-x 1 root root 48 May 28 2017 [01;32mget_ipaddr.sh[0m -rw-r--r-- 1 root root 1671 Jan 23 2018 prepare.sh
Yes indeed, it copied the files
Lets see if we can copy the whole directory now with files
rsync -ztva /home/downloads/home/scripts . sending incremental file list scripts/ scripts/get_free_memory.sh scripts/get_ipaddr.sh scripts/prepare.sh sent 1,076 bytes received 77 bytes 2,306.00 bytes/sec total size is 1,766 speedup is 1.53
Now we see the scripts directory copied too, the difference is the trailing trash, If you dont specify the trailing slash, it copies the directory too. Lets check our scripts directory now in /tmp/
pwd;ls -lrt /tmp/scripts /tmp total 12 -rwxr-xr-x 1 root root 47 May 28 2017 [0m[01;32mget_free_memory.sh[0m -rwxr-xr-x 1 root root 48 May 28 2017 [01;32mget_ipaddr.sh[0m -rw-r--r-- 1 root root 1671 Jan 23 2018 prepare.sh
Ok so far so good.
Similarly we can copy one remote server too, only difference is that for source and destination you need to follow the ssh syntax.
Lets say we want to copy the scripts directory which is under the same path but on server2.
Lets copy it from server2 to current /tmp/ directory
rsync -ztva [email protected]:/home/downloads/home/scripts /tmp/
There are many other commands which rsync provides. I just touched few of them above.
How to use grsync
Now I want to talk about another great Utility grsync, which is GUI utility for rsync.
It works on all the major platforms, such as Centos, Ubuntu, Windows and Mac.
To install on Centos do...
yum install grsync
Once you have it installed, You can invoke the GUI by just typing grsync. It has pretty simple GUI interface which is self explanatory.
One of the great benefit of using grsync is that it can be used to syncup folders between two servers. Here is how it works.
- Create a new session by going sessions > add
- A new pop up window will show up, Give your session a name.
- Then fill up the information for source and destination servers. Then click simulation. It will ask for password on the command prompt.
- If all the settings are good, Then it will actually simulate rsync operation and show you the output. If you liked the output, you can click execute to fulfill the operation.
Example: I created a session named "copy_from_server2_current_server" Here is the output of the simulate function...
**** copy_from_server2_current_server - Thu Aug 22 00:43:28 2019 ** Launching RSYNC command (simulation mode): rsync -r -n -t -v --progress -s [email protected]:/home/downloads/home /tmp/ receiving incremental file list home/ home/scripts/ home/scripts/get_free_memory.sh home/scripts/get_ipaddr.sh home/scripts/prepare.sh sent 41 bytes received 162 bytes 27.07 bytes/sec total size is 1,766 speedup is 8.70 (DRY RUN) Rsync process exit status: 0
In the simulation it showed me what command it would use...
rsync -r -n -t -v --progress -s [email protected]:/home/downloads/home /tmp/
The great benefit of using grsync is that, you chose all the options through GUI and you dont need to remember these options.
Another benefit is that We can call this session in a cron job and setup a sync up between two directories such as this...
0 11 * * * grsync -e "copy_from_server2_current_server"
Above cron will run the copy_from_server2_current_server rsync command using grsync 11 o'clock everyday.
That's it for now.