Find Duplicate Files Script

Overview

Find duplicate files using cryptographic hashes. Supports multiple actions: report, delete, move to folder, or create hard links to save space.

The Script

#!/bin/bash
# Find Duplicate Files
# Usage: ./find-dupes.sh [options] <directory>

set -euo pipefail

HASH_CMD="md5sum"
ACTION="report"
MIN_SIZE=1

echo "Scanning files..."

# Find files and compute hashes
find "$SEARCH_DIR" -type f -size +\${MIN_SIZE}c -print0 2>/dev/null | \\
while IFS= read -r -d '' file; do
    $HASH_CMD "$file" 2>/dev/null
done | sort > "$HASH_FILE"

echo "Done!"

Usage Examples

# Report duplicates
./find-dupes.sh ~/Downloads

# Use SHA256 for more accuracy
./find-dupes.sh -a sha256 ~/Photos

# Only check files larger than 1MB
./find-dupes.sh -s 1000000 ~/Videos

# Delete duplicates (keep first found)
./find-dupes.sh -d ~/Documents

# Move duplicates to separate folder
./find-dupes.sh -m ~/Duplicates ~/Downloads

# Replace duplicates with hard links (saves space)
./find-dupes.sh -l ~/Photos