Copying Files from Region to Region Using RSYNC or FPSYNC is Slow

Using rsync or fpsync to copy data from a file system in one region to a file system in another region results in slow progress.

Cause: NFS traffic from region to region is generally expensive. File copies between two NFS servers, each in a different region, can take time.

Solution: Instead of using rsync or fpsync to copy data between mounted file systems in different regions, use fpsync and instance-to-instance streaming.

The fpsync tool is a parallel wrapper of rsync. To install fpsync, enable the Oracle Linux developer repository, which includes the fpsync utility, on an OCI instance using a command such as the following:

yum --enablerepo ol7_developer_EPEL install -y fpart

yum --enablerepo ol8_developer_EPEL install -y fpart

A standard command such as this copies data:

fpsync /<fss_src_region>/test /<fss_dest_region>/

After installing the tool, use instance-to-instance streaming and a command such as this:

fpsync -o "-e ssh --progress" /<fss_src_region>/test <ssh_user>@<remote_ip>:/<fss_dest_region>/

For more information and options, see the fpsync man page.

# date; time fpsync -o "-e ssh --progress --log-file ~/speedtest.log" /fss_src_bom/test root@DR_hyd:/fss_dest_hyd/ ; date
Sun Mar 13 15:22:58 GMT 2022

real 0m1.467s
user 0m0.111s
sys 0m0.075s
Sun Mar 13 15:23:00 GMT 2022

# ls -ltrd test
drwxr-xr-x. 2 root root 1 Mar 13 15:22 test
# du -sh test
1001M test
# cp -r test test1

# date; time fpsync -o "--progress --log-file ~/speedtest1.log" /fss_src_bom/test1 /fss_dest_hyd/ ; date
Sun Mar 13 15:25:16 GMT 2022

real 1m28.847s
user 0m3.688s
sys 0m1.439s
Sun Mar 13 15:26:44 GMT 202