rsync is a free-to-use command-line tool that lets you sync files locally and remotely. With it, you can transfer files and directories.

This makes it very useful for migrating data, performing backups, and mirroring, making it great for webmasters and administrators for recurring data transfer tasks.

This article will teach us how Rysnc works and give some helpful examples.

How Does rsync Work?

rsync is a remote synchronization command-line utility for Unix-like systems. It gives the user the ability to transfer/sync files between machines or two hosts seamlessly.

Under the hood, rsync uses the delta-transfer algorithm. This algorithm involves copying differences between the two files across two hosts (source and destination). The source and destination can be local or remote. The rsync command can carry out copy/sync data tasks in two ways. These are:

  • Use ssh, rsh to copy/sync between two hosts.
  • Use TCP rsync daemon for copying/syncing.

As it is designed to work in Unix-like systems, it also works in Linux. 

The step-by-step process rsync uses as follows:

  • Rsync uses SSH to connect to the remote host and asks for a password.
  • Once connected, the remote host’s rsync communicates with the source.
  • These programs will then determine the files and directories that need sync. It uses a delta-transfer algorithm to check the difference between the files.
image-159

If the rsync command doesn’t find a file on the remote but not on the host, it’ll copy and transfer it to the host system. Similarly, for already existing files (both on host and remote), it’ll copy the difference between them (i.e., changed parts). Lastly, it’ll ignore existing files on both systems with no modifications.

The rsync command achieves incremental updates by storing the difference in a temporary file before pushing it to the destination.

Before you start:

  • You need to have root or sudo privileges
  • Have access to the terminal/command line
  • SSH access for securely running rsync commands
  • You also need two machines to make rsync. 

For the tutorial, we’re going to use Linux Mint 21.1 Vera. You can easily follow the guide without issues using any other Linux distro.

Why Use rsync Over Scp?

Scp (secure copy) is a popular command-line protocol that lets users copy files. It follows a linear approach to copy. This means it simply copies the files from source to destination. To securely do so, it uses SSH.

The scp command syntax is as below:

scp option SOURCE DESTINATION

However, the question remains, why use rsync over scp?

Well, there’re multiple benefits of using rsync. These benefits include:

  • Faster Execution: rsync is faster than scp because it uses a remote-update protocol. This allows it to transfer only differences rather than the whole file. So, if you set up a regular sync over time, you only need to do a full copy the first time. From the next time onwards, it only copies the updated parts with the help of the delta-transfer algorithm.
  • Less Bandwidth Consumption: As rsync doesn’t have to copy the whole file again, it leads to less bandwidth consumption. Moreover, it also uses compression/decompression algorithms to minimize file size during transfer.
  • More Command-Line Options: rsync also beats scp in the number of command-line options. This means you can fine-tune rsync more than scp.

In conclusion, rsync is an excellent incremental sync tool. The scp tool is also helpful for securely transferring/syncing files. In a real-world scenario, scp is suitable for day-to-day tasks if you’re looking for a more straightforward approach. However, for recurring tasks, use rsync.

rsync Command Syntax

The rsync command is as below:

  • Local to Local: rsync option SOURCE DEST
  • Local to Remote: rsync option SOURCE USER@HOST:DEST
  • Remote to Local: rsync option USER@HOST:SRC DEST

Here, the terms are defined below:

  • OPTION – These are the rsync options
  • SOURCE – Source directory
  • DEST – Destination directory
  • USER – Remote username
  • HOST – Remote hostname or IP address

The options are parameters that offer additional functionality to the command. For example, you can sync data recursively with the -r option. However, the -r option doesn’t sync ownership for groups and users, timestamps, permission, or symbolic links. Instead, you can use the -a option, which runs the command in archive mode, ensuring that all the file’s ownership, permission, and symbolic links are kept when copying.

Other rsync options include:

  • -z: compress data to save space
  • -h: provides human-readable format output
  • -b: performs backup during the data synchronization process
  • -e: use SSH protocol for remote data transfers
  • -progress: display the data synchronization progress
  • -v: ask rsync to display verbose output
  • –n: carry out a dry run to test out settings and setup for data synchronization
  • -q: suppress rsync command output and options

Installing rsync

Most Unix and Linux systems come with rsync installed. If your system doesn’t have rsync, you can install it using the following command(s).

For Debian/Ubuntu and Mint

sudo apt-get install rsync

For Arch Linux

pacman -S rsync

On Gentoo

emerge sys-apps/rsync

On CentOS/Fedora/REHL

sudo yum install rsync

On openSUSE

sudo zypper install rsync

As we have Linux Mint, rsync came pre-installed. So you can run the rsync command in the terminal to see if it is installed. It’ll explain its capabilities, options, and other vital information if it is present.

nitt ~
$ rsync
rsync  version 3.2.3  protocol version 31
Copyright (C) 1996-2020 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
    64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, hardlink-specials, symlinks, IPv6, atimes,
    batchfiles, inplace, append, ACLs, xattrs, optional protect-args, iconv,
    symtimes, prealloc, stop-at, no crtimes
Optimizations:
    SIMD, no asm, openssl-crypto
Checksum list:
    xxh128 xxh3 xxh64 (xxhash) md5 md4 none
Compress list:
    zstd lz4 zlibx zlib none

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

rsync is a file transfer program capable of efficient remote update
via a fast differencing algorithm.

Let’s now see rsync in action with different use cases.

For the tutorial, we’re going to make two directories.

  • source-directory
  • dest-directory

For that, you need to type the following commands.

mkdir source-directory
mkdir dest-directory

Also, we’ll need to generate files to make rsync work. So let’s create some test files with the command below.

touch source-directory/file{1..100}

This creates 100 empty files in the source directory. You can confirm it by entering the command below.

ls source-directory
Output:
nitt rsync-tutorial
$ ls
dest-directory  source-directory
nitt rsync-tutorial
$ touch source-directory/file{1..100}
nitt rsync-tutorial
$ ls source-directory
file1    file18  file27  file36  file45  file54  file63  file72  file81  file90
file10   file19  file28  file37  file46  file55  file64  file73  file82  file91
file100  file2   file29  file38  file47  file56  file65  file74  file83  file92
file11   file20  file3   file39  file48  file57  file66  file75  file84  file93
file12   file21  file30  file4   file49  file58  file67  file76  file85  file94
file13   file22  file31  file40  file5   file59  file68  file77  file86  file95
file14   file23  file32  file41  file50  file6   file69  file78  file87  file96
file15   file24  file33  file42  file51  file60  file7   file79  file88  file97
file16   file25  file34  file43  file52  file61  file70  file8   file89  file98
file17   file26  file35  file44  file53  file62  file71  file80  file9   file99

Note: We have kept the dest-directory empty.

Copy Directories Locally Recursively

Even though rsync primarily aims to copy files between a local and remote server, it is equally helpful for copying files locally. 

In this case, we’ll use the following rsync syntax.

rsync option SOURCE DESTINATION

Copy the files in our source-directory to the dest-directory by running the following command.

rsync -r source-directory/ dest-directory

Here, the command “recursively” copies the files from the source-directory to the dest-directory.

nitt rsync-tutorial
$ ls
dest-directory  source-directory
nitt rsync-tutorial
$ ls dest-directory
nitt rsync-tutorial
$ ls source-directory
file1    file18  file27  file36  file45  file54  file63  file72  file81  file90
file10   file19  file28  file37  file46  file55  file64  file73  file82  file91
file100  file2   file29  file38  file47  file56  file65  file74  file83  file92
file11   file20  file3   file39  file48  file57  file66  file75  file84  file93
file12   file21  file30  file4   file49  file58  file67  file76  file85  file94
file13   file22  file31  file40  file5   file59  file68  file77  file86  file95
file14   file23  file32  file41  file50  file6   file69  file78  file87  file96
file15   file24  file33  file42  file51  file60  file7   file79  file88  file97
file16   file25  file34  file43  file52  file61  file70  file8   file89  file98
file17   file26  file35  file44  file53  file62  file71  file80  file9   file99
nitt rsync-tutorial
$ rsync -r source-directory/ dest-directory
nitt rsync-tutorial
$ ls dest-directory
file1    file18  file27  file36  file45  file54  file63  file72  file81  file90
file10   file19  file28  file37  file46  file55  file64  file73  file82  file91
file100  file2   file29  file38  file47  file56  file65  file74  file83  file92
file11   file20  file3   file39  file48  file57  file66  file75  file84  file93
file12   file21  file30  file4   file49  file58  file67  file76  file85  file94
file13   file22  file31  file40  file5   file59  file68  file77  file86  file95
file14   file23  file32  file41  file50  file6   file69  file78  file87  file96
file15   file24  file33  file42  file51  file60  file7   file79  file88  file97
file16   file25  file34  file43  file52  file61  file70  file8   file89  file98
file17   file26  file35  file44  file53  file62  file71  file80  file9   file99

This copies every single file in the source-directory to the dest-directory.

Copy Single-File Locally

Now that we have learned how to sync two directories, we’ll learn how to copy a single file.

For this purpose, we’ll edit the empty file file1 in the source-directory and then sync it with the same file present in the dest-directory.

To edit a file, type the following command.

nano source-directory/file1

Now copy-paste the following text into the nano editor.

We're updating file1 in the source-directory.
Once we update, we'll push the new update to the dest-directory.
rsync doesn't need to re-copy the file using the delta-transfer algorithm.
This algorithm checks the differences and then updates destination files
accordingly.
So, once you save file1 in the source-directory, open file1 in
the dest-directory. It'll be empty.
However, if we check it after running rsync, you'll see that file1 in dest-directory
is updated.
updating-file1-with-text
Adding content to a file

Now, close and save the file.

Let’s now copy the file using rsync.

sudo rsync -v --existing source-directory/file1 /dest-directory
#Output
nitt rsync-tutorial
$ sudo rsync -v --existing source-directory/file1 dest-directory
file1

sent 557 bytes  received 35 bytes  1,184.00 bytes/sec
total size is 474  speedup is 0.80

nitt rsync-tutorial
$

As you might have noticed, we used –existing option. This updates the existing files. If you run the command without –existing, it’ll not update, even though it runs successfully.

If the single file you’re copying doesn’t exist in the destination folder, you need to run the following command.

rsync -v source-directory/newfile dest-directory
#Output
nitt rsync-tutorial
$ rsync -v source-directory/newfile dest-directory
newfile

sent 82 bytes  received 35 bytes  234.00 bytes/sec
total size is 0  speedup is 0.00

Copying Multiple Files

If you intend to copy multiple files, you must mention both files’ source paths, followed by the target directory.

For this purpose, we’ll create two new files: newfile2 and newfile3, in the source-directory. Run the following command to do so.

touch source-directory/newfile{2..3}

Now, use the following command to sync multiple files.

rsync -v source-directory/newfile2 source-directory/newfile3 dest-directory
#Output
nitt rsync-tutorial
$ rsync -v source-directory/newfile2 source-directory/newfile3 dest-directory
newfile2
newfile3

sent 135 bytes  received 54 bytes  378.00 bytes/sec
total size is 0  speedup is 0.00
nitt rsync-tutorial
$

Copy File or Directory From Local to Remote

The rsync syntax for local to remote transfer is as below.

rsync option SOURCE USER@HOST:DEST

As you can see, you need to know the IP address of the remote machine to make this work. Also, you need to add the IP address after the source file.

rsync -av /Projects/rsync-tutorial/source-directory 192.168.192.200: /Projects/rsync-tutorial/dest-directory

Before starting the transfer, it’ll ask for the password. Enter it to continue. Also, if the user is different in the remote machine, then you need to mention it before the IP address followed by @.

rsync -av /Projects/rsync-tutorial/source-directory nithya@192.168.192.200: /Projects/rsync-tutorial/dest-directory

If you intend to copy multiple files or directories, then you need to list the files or directory paths. The syntax for it is below.

rsync -av /Projects/rsync-tutorial/source-directory/newfile1 /Projects/rsync-tutorial/source-directory/newfile2 192.168.192.200: /Projects/rsync-tutorial/dest-directory

Remote Transfer-Specific Protocols

rsync lets you specify specific file transfer protocols. To do so, you need to use the -e option, followed by the protocol.

For example, if you want to use the ssh protocol, you must append -e ssh to the rsync command.

rsync -e ssh /Projects/rsync-tutorial/source-directory 192.168.192.200: /Projects/rsync-tutorial/dest-directory

Copying a File or Directory From Remote to Local

The rsync syntax for local to remote transfer is as below.

sync option USER@HOST:SRC DEST

This acts as a pull request. So, to get the required files/directory from a remote server to your local machine, you need to run the following command.

rsync -av 192.168.192.200: /Projects/rsync-tutorial/dest-directory /Projects/rsync-tutorial/source-directory

The command pulls the files within the remote machine’s dest-directory to the local machine’s source-directory.

Similarly, you can pull a specific file by mentioning it through its full path.

rsync -av 192.168.192.200: /Projects/rsync-tutorial/dest-directory/newfile4 /Projects/rsync-tutorial/source-directory

To copy multiple files or directories from local to remote, you need to specify the path within the curly brackets (separated by commas) after the server IP address.

rsync -av 192.168.192.200: {/Projects/rsync-tutorial/dest-directory/, /home/music/2023-music} /Projects/rsync-tutorial/source-directory

Similarly, you can mention files as well.

Show Live Progress During Transfer

It is a good idea to see live progress for larger data backups. For that, you need to use the –progress flag. Append it to the rsync command, and you can see the transfer speed, remaining time, and amount of data transferred.

rsync -av --progress 192.168.192.200: /Projects/rsync-tutorial/dest-directory /Projects/rsync-tutorial/source-directory

Delete Source Files Once the Transfer Is Complete

You can use the -remove-source-files flag to remove source files after the complete transfer. This can help you in many scenarios, especially when it comes to securing your files without leaving any trace or simply using them to free up space.

rsync -av --remove-source-files 192.168.192.200: /Projects/rsync-tutorial/dest-directory /Projects/rsync-tutorial/source-directory

rsync dry run

Rysnc also lets you evaluate before you do an actual run. The dry run lets you see if you’re doing the right thing. After all, you don’t want to update files by mistake or even delete them.

You need to use the –dry-run option to do a dry run. Add it to the command, followed by the source and destination.

sudo rsync -v --dry-run source-directory/file1 dest-directory
nitt rsync-tutorial
$ sudo rsync -v --dry-run source-directory/file1 dest-directory
file1

sent 43 bytes  received 19 bytes  124.00 bytes/sec
total size is 474  speedup is 7.65 (DRY RUN)
nitt rsync-tutorial

The output is similar to how we ran it before. However, you’ll notice a mention of (DRY RUN) at the end of the output.

Set Minimum and Maximum File Size

Rysnc also lets you set minimum or maximum file size during transfer.

So, if you want to transfer with a minimum size of 15KB, you need to use –min-size=15K.

rsync -av --min-size=15k 192.168.192.200: /Projects/rsync-tutorial/dest-directory /Projects/rsync-tutorial/source-directory

The command will only copy files with a minimum file size of 15KB. If the file size is below, it’ll ignore it.

Similarly, you can use the –max-size flag to set the maximum file size.

rsync -av --max-size=450k 192.168.192.200: /Projects/rsync-tutorial/dest-directory /Projects/rsync-tutorial/source-directory

Here, rsync will ignore files above the 450K size.

Set Bandwidth Limit

If you’re doing other bandwidth-intensive tasks, you can set rysnc to set the bandwidth limit for the transfer. To do so, use the –bwlimit=KB/s.

rsync -av --bwlimit=100 --progress 192.168.192.200: /Projects/rsync-tutorial/dest-directory /Projects/rsync-tutorial/source-directory

Here, we set the bandwidth to 100Kb/s during the transfer.

Final Words

This leads us to the end of our rsync command guide. We learned how to use rysnc and covered plenty of commands effectively.

However, rysnc offers way more than what we covered. That’s why you may also want to go through the rsync man page, where every aspect of rsync is covered.

Next, check out command line cheat sheets for Windows, Linux, and macOS.