Troubleshooting

chevron-rightcommon issues and mitigationshashtag

ssh to managed nodes via bastion host check kubelet status

ssh -i vg-karp-ssh.pem [email protected]

kubectl get pods --all-namespaces -o wide | grep ip-10-1-4-117

drain node so that pod can be schedule on helathy node

pod capacity of server 17 pod per node for t3a.medium

for checking cpu and memory consuption check cpu and memory via metric server kubectl top node

hpa need prerequisites as metric server

eks uses containerd as CRI plugin VPC CNI plugin EBS CSI driver for storage

total pod running on eks kubectl get pods --all-namespaces --no-headers | wc -l

pod limit because of vpc cni depend on ec2 type as t3a.medium have 17 pod limit

check nodes aws eks list-nodegroups aws eks describe-nodegroup --cluster-name VG-KARP-CLUSTER

eks/k8s event

kubectl get events --all-namespaces --sort-by=.metadata.creationTimestamp

Kubernetes events are ephemeral. By default:

Retention is very short (usually ~1 hour in EKS).

Older events get garbage-collected automatically.

kubectl get events only shows events currently stored in etcd, so anything older than that is gone.

dashboard to build for k8s

total nodes nodes cpu, memory, disk usage each node capacity totla cluster pod capacity total used pods

karpenter events kubelet status

check pod

kubectl get pods -A

show pod status - running/pending check did it restarted

cluster access aws eks update-kubeconfig --name VG-KARP-CLUSTER --region ap-southeast-1 make sure aws configure user have access to clusyter

StatefulSets are mostly immutable. You cannot change fields like: volumeClaimTemplates (size, storageClass) after creation Certain spec fields other than replicas, template, updateStrategy, minReadySeconds Helm tries to apply your values.yaml → Kubernetes rejects it because the StatefulSet spec changed in an immutable way.

AllowVolumeExpansion: → behaves as false

your current ebs-csi StorageClass is missing

every pod have service account but it does not have cluster level permission check what default pod service account have permission

Last updated