HxHippy

Disk & Storage Issues

Diagnose full disks, I/O problems, and mount issues.

Last updated: 2025-01-15

Disk & Storage Issues

Diagnose and fix disk, storage, and filesystem problems.

Quick Diagnostics

# Disk usage
df -h
df -i  # Inodes

# Largest directories
du -sh /* 2>/dev/null | sort -h | tail -10

# I/O status
iostat -x 1 3

Disk Full Issues

Find Large Files

# Largest files
find / -type f -size +100M -exec ls -lh {} \; 2>/dev/null | sort -k5 -h

# Largest directories
du -h --max-depth=1 / 2>/dev/null | sort -h | tail -20

# Top 10 in specific directory
du -sh /var/* | sort -h | tail -10

Common Culprits

# Log files
du -sh /var/log/*

# Package cache
du -sh /var/cache/apt/archives

# Docker
docker system df
docker system prune -a

# Old kernels
dpkg -l 'linux-*' | grep ^ii

Quick Cleanup

# Clear package cache
sudo apt clean
sudo apt autoremove

# Clear old logs
sudo journalctl --vacuum-size=100M
sudo find /var/log -type f -name "*.gz" -delete

# Clear tmp
sudo rm -rf /tmp/*

Inode Exhaustion

# Check inodes
df -i

# Find directory with most files
find / -xdev -type d -exec sh -c 'echo "$(find "{}" -maxdepth 1 | wc -l) {}"' \; | sort -n | tail -20

# Common causes: mail queue, session files
find /var/spool -type f | wc -l
find /tmp -type f | wc -l

I/O Performance

Diagnose High I/O

# I/O wait
top  # Look at wa%
vmstat 1 5

# Per-disk stats
iostat -x 1 5

# Process I/O
iotop
pidstat -d 1 5

# Which process is causing I/O
sudo iotop -o

Identify I/O Hogs

# Processes doing I/O
sudo iotop -b -n 2

# Files being accessed
sudo lsof +D /path/to/dir

# Disk activity by process
sudo strace -p PID -e trace=read,write

Mount Issues

Check Mounts

# Current mounts
mount | column -t
cat /proc/mounts

# Check fstab
cat /etc/fstab

# Test mount without mounting
sudo mount -fav

Fix Mount Issues

# Remount as read-write
sudo mount -o remount,rw /

# Force unmount
sudo umount -f /mnt/point
sudo umount -l /mnt/point  # Lazy unmount

# Find what's using mount
sudo lsof +f -- /mnt/point
sudo fuser -vm /mnt/point

Filesystem Errors

# Check filesystem (unmount first!)
sudo fsck /dev/sda1

# Check without fixing
sudo fsck -n /dev/sda1

# Force check on boot
sudo touch /forcefsck

# Check for bad blocks
sudo badblocks -v /dev/sda1

SMART Status

# Install smartmontools
sudo apt install smartmontools

# Check disk health
sudo smartctl -H /dev/sda

# Full info
sudo smartctl -a /dev/sda

# Run self-test
sudo smartctl -t short /dev/sda

Common Issues

Issue Diagnostic Fix
Disk full df -h Remove large files/logs
Inodes full df -i Remove many small files
High I/O wait iostat, iotop Identify I/O hog
Read-only mount mount Check cable, fsck, remount
Mount failed dmesg Check filesystem, fstab
intermediate Troubleshooting Updated 2025-01-15
  • disk
  • storage
  • troubleshooting
  • io
  • filesystem