Log Analysis and Management

Log Analysis Guide

Effective logging is essential for security monitoring and incident response.

System Logs Overview

Key Log Files

journalctl (systemd)

# View all logs
journalctl

# Follow logs in real-time
journalctl -f

# Logs since boot
journalctl -b

# Logs by unit
journalctl -u sshd.service

# Logs by priority
journalctl -p err   # Errors and above
journalctl -p warning

# Logs by time
journalctl --since "2024-01-01" --until "2024-01-02"
journalctl --since "1 hour ago"

# JSON output
journalctl -o json-pretty

Security-Focused Log Analysis

SSH Monitoring

# Failed SSH attempts
grep "Failed password" /var/log/auth.log | tail -20

# Successful logins
grep "Accepted" /var/log/auth.log

# Invalid users
grep "Invalid user" /var/log/auth.log

# Summary of IPs
grep "Failed password" /var/log/auth.log | \
    awk '{print $(NF-3)}' | sort | uniq -c | sort -rn

sudo Usage

# All sudo usage
grep sudo /var/log/auth.log

# Failed sudo attempts
grep "authentication failure" /var/log/auth.log | grep sudo

Log Rotation

# /etc/logrotate.d/rsyslog
/var/log/syslog
/var/log/auth.log
{
    rotate 52
    weekly
    missingok
    notifempty
    compress
    delaycompress
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
}

Centralized Logging

rsyslog Remote Logging

# Server: /etc/rsyslog.conf
module(load="imtcp")
input(type="imtcp" port="514")

# Client: /etc/rsyslog.d/50-remote.conf
*.* @@logserver.example.com:514

Log Analysis Tools

# lnav - Log file navigator
lnav /var/log/auth.log

# logwatch - Daily log summary
sudo apt install logwatch
logwatch --detail High --range today

Security Log Monitoring Script

#!/bin/bash
# Daily security log check

echo "=== Failed SSH Logins ==="
grep "Failed password" /var/log/auth.log | wc -l

echo "=== Root logins ==="
grep "session opened for user root" /var/log/auth.log

echo "=== sudo failures ==="
grep "authentication failure" /var/log/auth.log | wc -l