Example

🔹 Full-Time Monitoring of AWS Resources Using CloudWatch

This guide covers: ✅ Setting up AWS CloudWatch for real-time monitoring ✅ Configuring alerts for logs and metrics ✅ Best practices for monitoring EC2, EKS, S3, and RDS ✅ Real-time example with notifications (SNS, Slack, Email)

---

🔹 Overview of AWS CloudWatch Monitoring

Amazon CloudWatch is a monitoring and observability service for AWS resources. It provides: ✅ Metrics: CPU, Memory, Disk Usage, Network Traffic, etc. ✅ Logs: Application logs, system logs, custom logs from AWS services. ✅ Alarms: Automated notifications when a metric crosses a threshold. ✅ Dashboards: Centralized visualization of all AWS services.

---

🔹 Step-by-Step Setup for Full-Time Monitoring

1️⃣ Enable CloudWatch Monitoring for AWS Resources

AWS services like EC2, RDS, and Lambda automatically send logs/metrics to CloudWatch, but some services need manual configuration.

1.1 Enable Detailed Monitoring for EC2

By default, EC2 instances send metrics every 5 minutes. To get 1-minute metrics:

1. Go to AWS Console → EC2 Dashboard

2. Select your EC2 instance

3. Click Actions → Monitor and Troubleshoot → Manage Detailed Monitoring

4. Click Enable Detailed Monitoring

✅ Common Issue: "I don’t see Memory usage in CloudWatch." 🔹 Fix: Memory & disk usage require CloudWatch Agent (see next step).

---

1.2 Install CloudWatch Agent for Memory & Disk Monitoring

sudo yum install amazon-cloudwatch-agent -y
sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard
sudo systemctl enable amazon-cloudwatch-agent
sudo systemctl start amazon-cloudwatch-agent

✅ Common Issue: "CloudWatch Agent not sending data." 🔹 Fix: Verify IAM role has CloudWatchAgentServerPolicy attached.

---

1.3 Enable CloudWatch Logging for Lambda Functions

1. Go to AWS Console → Lambda

2. Select your function → Configuration

3. Click Monitor → Enable CloudWatch Logs

✅ Common Issue: "Lambda logs not appearing in CloudWatch." 🔹 Fix: Ensure the IAM Role attached to Lambda has AWSLambdaBasicExecutionRole.

---

2️⃣ Setting Up Alarms for AWS Metrics

2.1 Create a CloudWatch Alarm for High CPU Usage on EC2

1. Go to AWS Console → CloudWatch → Alarms → Create Alarm

2. Click Select metric → EC2 Metrics

3. Select CPUUtilization for your instance

4. Set Threshold type: Greater than 80%

5. Configure Actions:

   • Notification: Send alert via SNS (email, Slack, etc.)

   • Auto Scaling: Add instances when CPU is high

6. Click Create Alarm

✅ Common Issue: "Alarm not triggering." 🔹 Fix: Ensure EC2 has detailed monitoring enabled.

---

2.2 Create an Alarm for Low Free Memory on EKS Nodes

1. Go to AWS Console → CloudWatch → Alarms → Create Alarm

2. Click Select metric → EKS Metrics

3. Select node_memory_MemFree_bytes

4. Set Threshold: Less than 500MB

5. Configure Actions: Send notification via SNS

✅ Common Issue: "EKS does not report memory usage." 🔹 Fix: Install CloudWatch Agent on worker nodes.

---

3️⃣ Monitoring Logs in CloudWatch & Setting Alerts

3.1 Enable CloudWatch Logs for EC2

1. Install the awslogs package:

   sudo yum install -y awslogs
   sudo systemctl start awslogsd

2. Configure /etc/awslogs/awslogs.conf:

   [general]
   state_file = /var/lib/awslogs/agent-state
   
   [/var/log/messages]
   file = /var/log/messages
   log_group_name = /var/log/messages
   log_stream_name = {instance_id}

3. Restart the service:

   sudo systemctl restart awslogsd

✅ Common Issue: "EC2 logs not appearing in CloudWatch." 🔹 Fix: Attach CloudWatchAgentServerPolicy to the EC2 IAM role.

---

3.2 Set Up Log-Based Alert for Failed SSH Logins

1. Go to CloudWatch → Logs → Create Metric Filter

2. Select Log Group: /var/log/secure

3. Enter filter pattern:

   [authentication failed]

4. Click Create Metric Filter

5. Create an Alarm with Threshold = 3 failed SSH attempts in 5 minutes

✅ Common Issue: "Metric filter not matching logs." 🔹 Fix: Ensure /var/log/secure contains relevant entries.

---

4️⃣ Setting Up CloudWatch Dashboards

4.1 Create a Custom Dashboard for Monitoring

1. Go to CloudWatch → Dashboards → Create Dashboard

2. Click Add Widget → Select Metrics

3. Choose EC2 CPU Usage, Memory, and Network Traffic

4. Save the dashboard

✅ Common Issue: "Metrics are not updating in the dashboard." 🔹 Fix: Ensure resources are reporting to CloudWatch.

---

5️⃣ Sending Alerts to Email & Slack

5.1 Create an SNS Topic for Notifications

1. Go to AWS Console → SNS → Create Topic

2. Topic Name: CloudWatchAlerts

3. Create Subscription:

   • Protocol: Email or Lambda (for Slack integration)

   • Endpoint: Your email or webhook URL

✅ Common Issue: "SNS notifications not being sent." 🔹 Fix: Confirm subscription activation via email.

---

5.2 Integrate CloudWatch Alarms with Slack

1. Create an SNS Topic as described above.

2. Use AWS Lambda to send messages to Slack:

import json
import urllib3

def lambda_handler(event, context):
    url = "https://hooks.slack.com/services/your-slack-webhook"
    message = {
        "text": "🚨 CloudWatch Alarm Triggered: " + json.dumps(event)
    }
    
    http = urllib3.PoolManager()
    response = http.request("POST", url, body=json.dumps(message).encode('utf-8'), headers={'Content-Type': 'application/json'})

    return {
        'statusCode': 200,
        'body': response.data
    }

✅ Common Issue: "Lambda not triggering SNS messages." 🔹 Fix: Ensure Lambda has SNS:Publish permission.

---

🔹 Summary of CloudWatch Full-Time Monitoring

✔ Enabled EC2, EKS, and Lambda monitoring ✔ Configured CloudWatch Agent for detailed metrics ✔ Set up alarms for CPU, Memory, and SSH login failures ✔ Created dashboards for real-time tracking ✔ Configured SNS alerts for email & Slack notifications

---

🔥 Next Steps

Would you like to automate incident response using AWS Lambda and Auto Scaling? Let me know! 🚀