Article 4WEZQ Borg backup data integrity error in a CentOS 7 VM

Borg backup data integrity error in a CentOS 7 VM

by
Superspeed500
from LinuxQuestions.org on (#4WEZQ)
I have a CentOS 7 Virtual machine (QEMU/KVM) running on top of a Ubuntu 18.04 server. The Ubuntu machine itself runs a shell script 2 o'clock in the night that uses borg backup. That script works without any problems. A similar script runs at 1 o'clock in the virtual machine. That script have started giving the following error.

Code:Data integrity error: Segment entry checksum mismatch [segment 1158, offset 48099855]
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 4501, in main
exit_code = archiver.run(args)
File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 4433, in run
return set_ec(func(args))
File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 153, in wrapper
kwargs['manifest'], kwargs['key'] = Manifest.load(repository, compatibility)
File "/usr/lib64/python3.6/site-packages/borg/helpers.py", line 365, in load
cdata = repository.get(cls.MANIFEST_ID)
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 1070, in get
self.index = self.open_index(self.get_transaction_id())
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 376, in get_transaction_id
self.check_transaction()
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 373, in check_transaction
self.replay_segments(replay_from, segments_transaction_id)
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 812, in replay_segments
self._update_index(segment, objects)
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 822, in _update_index
for tag, key, offset, size in objects:
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 1378, in iter_objects
read_data=read_data)
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 1476, in _read
segment, offset))
borg.helpers.IntegrityError: Data integrity error: Segment entry checksum mismatch [segment 1158, offset 48099855]

Platform: Linux scotty.speednet.lan 3.10.0-1062.7.1.el7.x86_64 #1 SMP Mon Dec 2 17:33:29 UTC 2019 x86_64
Linux: CentOS Linux 7.7.1908 Core
Borg: 1.1.10 Python: CPython 3.6.8 msgpack: 0.5.6
PID: 19390 CWD: /root
sys.argv: ['/bin/borg', 'create', '-s', '-C', 'zlib,6', '--one-file-system', '::root-{now:%Y-%m-%f}', '/', '/home', '/srv', '/var', '--exclude', '/dev', '--exclude', '/proc', '--exclude', '/tmp', '--exclude', '/run', '--exclude-caches', '--exclude', '/home/*/.cache/*', '--exclude', '/var/cache/*', '--exclude', '/var/tmp/*', '--exclude', '/mnt']
SSH_ORIGINAL_COMMAND: None

Data integrity error: Segment entry checksum mismatch [segment 1158, offset 48099855]
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 4501, in main
exit_code = archiver.run(args)
File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 4433, in run
return set_ec(func(args))
File "/usr/lib64/python3.6/site-packages/borg/archiver.py", line 153, in wrapper
kwargs['manifest'], kwargs['key'] = Manifest.load(repository, compatibility)
File "/usr/lib64/python3.6/site-packages/borg/helpers.py", line 365, in load
cdata = repository.get(cls.MANIFEST_ID)
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 1070, in get
self.index = self.open_index(self.get_transaction_id())
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 376, in get_transaction_id
self.check_transaction()
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 373, in check_transaction
self.replay_segments(replay_from, segments_transaction_id)
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 812, in replay_segments
self._update_index(segment, objects)
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 822, in _update_index
for tag, key, offset, size in objects:
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 1378, in iter_objects
read_data=read_data)
File "/usr/lib64/python3.6/site-packages/borg/repository.py", line 1476, in _read
segment, offset))
borg.helpers.IntegrityError: Data integrity error: Segment entry checksum mismatch [segment 1158, offset 48099855]

Platform: Linux scotty.speednet.lan 3.10.0-1062.7.1.el7.x86_64 #1 SMP Mon Dec 2 17:33:29 UTC 2019 x86_64
Linux: CentOS Linux 7.7.1908 Core
Borg: 1.1.10 Python: CPython 3.6.8 msgpack: 0.5.6
PID: 19395 CWD: /root
sys.argv: ['/bin/borg', 'prune', '--show-rc', '--list', '--keep-daily', '7', '--keep-weekly', '4', '--keep-monthly', '6']
SSH_ORIGINAL_COMMAND: None

terminating with error status, rc 2I can fix the problem temporary by running borg check --repair (https://borgbackup.readthedocs.io/en...age/check.html). I have also tried to delete the entire archive and then recreate it. That fixed the problem for a few days, but the problem is back again now.

The entire VM is backed up by the working backup script on the host. The VM is my LDAP server and my file server.

The backup script can be found bellow. The script is invoked using ionice in order to not disturb any other services while the backup is running. The target destination for the backup is my QNAP NAS. The protocol used is NFS.
Code:#! /bin/bash
# Server backup script
# Superspeed500 2019

export SHARE=/mnt/backup
export BORG_REPO=$SHARE/scotty
export BORG_PASSPHRASE="CENSORED"
export BORG_UNKNOWN_UNENCRYPTED_REPO_ACCESS_IS_OK=yes

# Mount the network share if not mounted
mountpoint -q $SHARE || mount $SHARE

# Do the backup
borg create -s -C zlib,6 --one-file-system ::root-{now:%Y-%m-%f} / /home /srv /var \
--exclude /dev \
--exclude /proc \
--exclude /tmp \
--exclude /run \
--exclude-caches \
--exclude '/home/*/.cache/*' \
--exclude '/var/cache/*' \
--exclude '/var/tmp/*' \
--exclude /mnt

# Prune archives
borg prune \
--show-rc \
--list \
--keep-daily 7 \
--keep-weekly 4 \
--keep-monthly 6

# Unset the password
unset BORG_PASSPHRASE

#EOFSo my questions are:
  • What can I do to fix the problem?
  • Should I rethink my VM backup strategy?
I appreciate any feedback on flaws in my backup strategy. I will also gladly post any additional information that is required to help me with this problem :)latest?d=yIl2AUoC8zA latest?i=cHcYMj6WU1A:fSIA-AYW6KI:F7zBnMy latest?i=cHcYMj6WU1A:fSIA-AYW6KI:V_sGLiP latest?d=qj6IDK7rITs latest?i=cHcYMj6WU1A:fSIA-AYW6KI:gIN9vFwcHcYMj6WU1A
External Content
Source RSS or Atom Feed
Feed Location https://feeds.feedburner.com/linuxquestions/latest
Feed Title LinuxQuestions.org
Feed Link https://www.linuxquestions.org/questions/
Reply 0 comments