RSYNC over SSH slow speed
by mackowiakp from LinuxQuestions.org on (#4WXAV)
I use such command to backup QNAP NAS to mounted on-demand (only for backup needs) USBv3 HDD connected to PC inside local LAN. The connection between QNAP and PC is done over 10 GB Ethernet. So network is not bottleneck.
This is related part of backup script:
Code:/opt/bin/rsync -qratx --delete --exclude-from=/share/homes/media/Pobrane/pomin_piotr.txt --force --progress -e '/opt/bin/ssh -x -q -i /share/homes/media/.ssh/id_media' /share/homes/media/Gosia_ISO/* root@$serwer:$backup_path/Gosia_ISOThe $backup_path/Gosia_ISO dir consist only 6 files but large:
Code:[root@Piotr ~]# ls -la /home/maciek/back/RSYNC/Gosia_ISO/
razem 12308364
drwxrwxr-x 2 toruser toruser 4096 gru 20 16:47 ./
drwxrwxrwx 23 root root 4096 gru 21 12:50 ../
-rw-rw-r-- 1 toruser toruser 6287774842 gru 20 05:52 gosia_sda5.img.gz
-rw-r--r-- 1 toruser toruser 81 gru 20 05:52 gosia_sda5.img.gz.md5
-rw-rw-r-- 1 toruser toruser 72909190 gru 20 06:05 gosia_sda7.img.gz
-rw-r--r-- 1 toruser toruser 81 gru 20 06:05 gosia_sda7.img.gz.md5
-rw-rw-r-- 1 toruser toruser 6243042497 gru 20 06:16 gosia_sda8.img.gz
-rw-r--r-- 1 toruser toruser 81 gru 20 06:17 gosia_sda8.img.gz.md5If I run above command, the sync speed is appx 5 M/s (as iotop stands). But when I first delete content of $backup_path/Gosia_ISO dir and then run rsync command, the throughput is appx 70 M/s.
Why the speed is so different in both cases? What to do to be the same without former deleting of $backup_path/Gosia_ISO dir content?


This is related part of backup script:
Code:/opt/bin/rsync -qratx --delete --exclude-from=/share/homes/media/Pobrane/pomin_piotr.txt --force --progress -e '/opt/bin/ssh -x -q -i /share/homes/media/.ssh/id_media' /share/homes/media/Gosia_ISO/* root@$serwer:$backup_path/Gosia_ISOThe $backup_path/Gosia_ISO dir consist only 6 files but large:
Code:[root@Piotr ~]# ls -la /home/maciek/back/RSYNC/Gosia_ISO/
razem 12308364
drwxrwxr-x 2 toruser toruser 4096 gru 20 16:47 ./
drwxrwxrwx 23 root root 4096 gru 21 12:50 ../
-rw-rw-r-- 1 toruser toruser 6287774842 gru 20 05:52 gosia_sda5.img.gz
-rw-r--r-- 1 toruser toruser 81 gru 20 05:52 gosia_sda5.img.gz.md5
-rw-rw-r-- 1 toruser toruser 72909190 gru 20 06:05 gosia_sda7.img.gz
-rw-r--r-- 1 toruser toruser 81 gru 20 06:05 gosia_sda7.img.gz.md5
-rw-rw-r-- 1 toruser toruser 6243042497 gru 20 06:16 gosia_sda8.img.gz
-rw-r--r-- 1 toruser toruser 81 gru 20 06:17 gosia_sda8.img.gz.md5If I run above command, the sync speed is appx 5 M/s (as iotop stands). But when I first delete content of $backup_path/Gosia_ISO dir and then run rsync command, the throughput is appx 70 M/s.
Why the speed is so different in both cases? What to do to be the same without former deleting of $backup_path/Gosia_ISO dir content?