Question about finding duplicate files using a bash script.
by Mithrilhall from LinuxQuestions.org on (#59GF1)
I found this script online and modified it. It appears to do what I want but I figured I would check before using.
I'm trying to use sha512sum to check for duplicate files in a directory and to move the duplicates to the "duplicates" directory.
Code:#!/bin/bash
#
# Usage: ./delete-duplicates.sh [<files...>]
#
declare -A filecksums
# No args, use files in current directory
test 0 -eq $# && set -- *
for file in "$@"
do
# Files only (also no symlinks)
[[ -f "$file" ]] && [[ ! -h "$file" ]] || continue
# Generate the checksum
cksum=$(sha512sum <"$file" | tr ' ' _)
# Have we already got this one?
if [[ -n "${filecksums[$cksum]}" ]] && [[ "${filecksums[$cksum]}" != "$file" ]]
then
echo "Found '$file' is a duplicate of '${filecksums[$cksum]}'" >&2
# echo mv -v "$file" ./duplicates
# echo mv "$file" ./duplicates
echo "$file" | mv "$file" ./duplicates
else
filecksums[$cksum]="$file"
fi
done


I'm trying to use sha512sum to check for duplicate files in a directory and to move the duplicates to the "duplicates" directory.
Code:#!/bin/bash
#
# Usage: ./delete-duplicates.sh [<files...>]
#
declare -A filecksums
# No args, use files in current directory
test 0 -eq $# && set -- *
for file in "$@"
do
# Files only (also no symlinks)
[[ -f "$file" ]] && [[ ! -h "$file" ]] || continue
# Generate the checksum
cksum=$(sha512sum <"$file" | tr ' ' _)
# Have we already got this one?
if [[ -n "${filecksums[$cksum]}" ]] && [[ "${filecksums[$cksum]}" != "$file" ]]
then
echo "Found '$file' is a duplicate of '${filecksums[$cksum]}'" >&2
# echo mv -v "$file" ./duplicates
# echo mv "$file" ./duplicates
echo "$file" | mv "$file" ./duplicates
else
filecksums[$cksum]="$file"
fi
done