Log Monitoring & Rotation

🚀 Log Monitoring & Rotation in Linux (Complete Guide)

In this guide, we'll cover real-time log monitoring, log rotation, and troubleshooting common log issues with explanations of what happens when you run each command.

---

🔍 Step 1: Checking Logs in Linux

1.1 Check System Logs

🔹 View system logs (recent logs & errors):

journalctl -xe

What Happens?

• Shows extended logs with errors, warnings, and critical messages.

• Useful for debugging service failures and system crashes.

🔹 View logs from a specific service (e.g., SSH):

journalctl -u sshd --since "1 hour ago"

What Happens?

• Filters logs for the sshd service in the last 1 hour.

🔹 Check kernel logs (boot & hardware issues):

dmesg | tail -20

What Happens?

• Displays last 20 kernel logs.

• Useful for troubleshooting boot issues.

---

1.2 Real-Time Log Monitoring

🔹 Monitor logs in real-time (live updates):

tail -f /var/log/syslog

What Happens?

• Continuously displays new log entries.

• Press CTRL + C to stop.

🔹 Monitor authentication logs (failed logins, SSH attempts):

tail -f /var/log/auth.log

What Happens?

• Shows login attempts and authentication failures.

🔹 Monitor specific service logs (e.g., Nginx):

tail -f /var/log/nginx/access.log

What Happens?

• Displays live access logs for Nginx.

✅ Common Issue: Logs Not Updating in Real-Time 🔹 Restart logging service:

systemctl restart systemd-journald

What Happens?

• Restarts system logging service to fix log display issues.

---

🛠️ Step 2: Log Rotation & Management

Linux systems generate a lot of logs. If logs are not rotated, they can fill up disk space and cause system slowdowns.

2.1 Checking Log Rotation Configuration

🔹 Check log rotation status:

logrotate -d /etc/logrotate.conf

What Happens?

• Runs a dry-run of log rotation (shows what it would do without actually rotating logs).

🔹 Check log rotation for a specific service (e.g., Apache):

cat /etc/logrotate.d/apache2

What Happens?

• Displays log rotation settings specific to Apache logs.

---

2.2 Manually Rotate Logs

🔹 Force log rotation immediately:

logrotate -f /etc/logrotate.conf

What Happens?

• Forces immediate rotation of logs based on the current config.

🔹 Compress logs after rotation:

gzip /var/log/syslog.1

What Happens?

• Compresses rotated logs to save space.

---

2.3 Common Log Rotation Issues & Fixes

✅ Issue: Logs Are Not Rotating 🔹 Check if logrotate service is running:

systemctl status logrotate

Solution: If not running, start it:

systemctl start logrotate

✅ Issue: Disk Full Due to Logs 🔹 Find large log files:

du -sh /var/log/*

🔹 Delete old logs manually (if needed):

rm -rf /var/log/*.gz

✅ Issue: Log Rotation Not Happening Automatically 🔹 Ensure logrotate is enabled in cron jobs:

cat /etc/cron.daily/logrotate

🔹 Manually trigger cron jobs:

run-parts /etc/cron.daily

---

🔎 Step 3: Advanced Log Monitoring (Bonus)

3.1 Using grep to Filter Logs

🔹 Find errors in logs:

grep -i "error" /var/log/syslog

What Happens?

• Searches for "error" (case-insensitive) in syslog.

🔹 Check failed SSH logins:

grep "Failed password" /var/log/auth.log

3.2 Using journalctl for Persistent Logs

🔹 View logs after reboot:

journalctl --boot

🔹 Clear old logs to free space:

journalctl --vacuum-time=7d

What Happens?

• Deletes logs older than 7 days.

---

🚀 Summary

✔ Checked system logs (journalctl, dmesg) ✔ Monitored logs in real-time (tail -f) ✔ Ensured log rotation (logrotate -d, gzip) ✔ Fixed common issues (disk full, log not rotating)

---

Next: Do you want to set up log monitoring with Prometheus + Loki + Grafana? 🚀