rsync is a free-to-use command-line tool that lets you sync files locally and remotely. With it, you can transfer files and directories.
This makes it very useful for migrating data, performing backups, and mirroring, making it great for webmasters and administrators for recurring data transfer tasks.
This article will teach us how Rysnc works and give some helpful examples.
How Does rsync Work?
rsync is a remote synchronization command-line utility for Unix-like systems. It gives the user the ability to transfer/sync files between machines or two hosts seamlessly.
Under the hood, rsync uses the delta-transfer algorithm. This algorithm involves copying differences between the two files across two hosts (source and destination). The source and destination can be local or remote. The rsync command can carry out copy/sync data tasks in two ways. These are:
Use ssh, rsh to copy/sync between two hosts.
Use TCP rsync daemon for copying/syncing.
As it is designed to work in Unix-like systems, it also works in Linux.
The step-by-step process rsync uses as follows:
Rsync uses SSH to connect to the remote host and asks for a password.
Once connected, the remote host’s rsync communicates with the source.
These programs will then determine the files and directories that need sync. It uses a delta-transfer algorithm to check the difference between the files.
If the rsync command doesn’t find a file on the remote but not on the host, it’ll copy and transfer it to the host system. Similarly, for already existing files (both on host and remote), it’ll copy the difference between them (i.e., changed parts). Lastly, it’ll ignore existing files on both systems with no modifications.
The rsync command achieves incremental updates by storing the difference in a temporary file before pushing it to the destination.
Before you start:
You need to have root or sudo privileges
Have access to the terminal/command line
SSH access for securely running rsync commands
You also need two machines to make rsync.
For the tutorial, we’re going to use Linux Mint 21.1 Vera. You can easily follow the guide without issues using any other Linux distro.
Why Use rsync Over Scp?
Scp (secure copy) is a popular command-line protocol that lets users copy files. It follows a linear approach to copy. This means it simply copies the files from source to destination. To securely do so, it uses SSH.
The scp command syntax is as below:
scp option SOURCE DESTINATION
However, the question remains, why use rsync over scp?
Well, there’re multiple benefits of using rsync. These benefits include:
Faster Execution: rsync is faster than scp because it uses a remote-update protocol. This allows it to transfer only differences rather than the whole file. So, if you set up a regular sync over time, you only need to do a full copy the first time. From the next time onwards, it only copies the updated parts with the help of the delta-transfer algorithm.
Less Bandwidth Consumption: As rsync doesn’t have to copy the whole file again, it leads to less bandwidth consumption. Moreover, it also uses compression/decompression algorithms to minimize file size during transfer.
More Command-Line Options: rsync also beats scp in the number of command-line options. This means you can fine-tune rsync more than scp.
In conclusion, rsync is an excellent incremental sync tool. The scp tool is also helpful for securely transferring/syncing files. In a real-world scenario, scp is suitable for day-to-day tasks if you’re looking for a more straightforward approach. However, for recurring tasks, use rsync.
rsync Command Syntax
The rsync command is as below:
Local to Local: rsync option SOURCE DEST
Local to Remote: rsync option SOURCE USER@HOST:DEST
Remote to Local: rsync option USER@HOST:SRC DEST
Here, the terms are defined below:
OPTION – These are the rsync options
SOURCE – Source directory
DEST – Destination directory
USER – Remote username
HOST – Remote hostname or IP address
The options are parameters that offer additional functionality to the command. For example, you can sync data recursively with the -r option. However, the -r option doesn’t sync ownership for groups and users, timestamps, permission, or symbolic links. Instead, you can use the -a option, which runs the command in archive mode, ensuring that all the file’s ownership, permission, and symbolic links are kept when copying.
Other rsync options include:
-z: compress data to save space
-h: provides human-readable format output
-b: performs backup during the data synchronization process
-e: use SSH protocol for remote data transfers
-progress: display the data synchronization progress
-v: ask rsync to display verbose output
–n: carry out a dry run to test out settings and setup for data synchronization
-q: suppress rsync command output and options
Most Unix and Linux systems come with rsync installed. If your system doesn’t have rsync, you can install it using the following command(s).
For Debian/Ubuntu and Mint
sudo apt-get install rsync
For Arch Linux
pacman -S rsync
sudo yum install rsync
sudo zypper install rsync
As we have Linux Mint, rsync came pre-installed. So you can run the rsync command in the terminal to see if it is installed. It’ll explain its capabilities, options, and other vital information if it is present.
rsync version 3.2.3 protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, hardlinks, hardlink-specials, symlinks, IPv6, atimes,
batchfiles, inplace, append, ACLs, xattrs, optional protect-args, iconv,
symtimes, prealloc, stop-at, no crtimes
SIMD, no asm, openssl-crypto
xxh128 xxh3 xxh64 (xxhash) md5 md4 none
zstd lz4 zlibx zlib none
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
rsync is a file transfer program capable of efficient remote update
via a fast differencing algorithm.
Let’s now see rsync in action with different use cases.
For the tutorial, we’re going to make two directories.
For that, you need to type the following commands.
Also, we’ll need to generate files to make rsync work. So let’s create some test files with the command below.
This creates 100 empty files in the source directory. You can confirm it by entering the command below.
This copies every single file in the source-directory to the dest-directory.
Copy Single-File Locally
Now that we have learned how to sync two directories, we’ll learn how to copy a single file.
For this purpose, we’ll edit the empty file file1 in the source-directory and then sync it with the same file present in the dest-directory.
To edit a file, type the following command.
Now copy-paste the following text into the nano editor.
We're updating file1 in the source-directory.
Once we update, we'll push the new update to the dest-directory.
rsync doesn't need to re-copy the file using the delta-transfer algorithm.
This algorithm checks the differences and then updates destination files
So, once you save file1 in the source-directory, open file1 in
the dest-directory. It'll be empty.
However, if we check it after running rsync, you'll see that file1 in dest-directory
It is a good idea to see live progress for larger data backups. For that, you need to use the –progress flag. Append it to the rsync command, and you can see the transfer speed, remaining time, and amount of data transferred.
You can use the -remove-source-files flag to remove source files after the complete transfer. This can help you in many scenarios, especially when it comes to securing your files without leaving any trace or simply using them to free up space.
I am a C1 Advanced(CEFR) certified writer with a master’s degree in computer science (B Level from NIELIT, India) with seven years of writing experience. My experience includes writing for the Web and covering diverse topics including Web3,… read more
Linux is a multiuser operating system thus, multiple users can interact with the same computer at the same time using Linux. Being a multiuser operating system, it is important for Linux to guarantee the security and privacy of the files belonging to different users.