Setting Up Cluster Autoscaler on EKS

Without Cluster Autoscaler, your EKS cluster has a fixed number of nodes. When pods can't be scheduled due to insufficient capacity, they stay Pending forever. With Cluster Autoscaler, AWS adds nodes automatically when needed and removes them when they're idle.

This tutorial sets it up correctly — including the IAM permissions that most guides gloss over.

How Cluster Autoscaler Works

The Cluster Autoscaler watches for unschedulable pods and checks whether adding a node would allow them to run. If yes, it increases the desired count of the matching Auto Scaling Group. It also watches for underutilised nodes and removes them after a configurable idle period.

Step 1: Tag Your Node Group ASG

Cluster Autoscaler discovers node groups by looking for specific tags on Auto Scaling Groups:

1# Get your ASG name
2aws autoscaling describe-auto-scaling-groups \
3  --query "AutoScalingGroups[?contains(Tags[?Key=='eks:cluster-name'].Value, 'my-cluster')].AutoScalingGroupName" \
4  --output text
5
6# Tag the ASG (replace values)
7aws autoscaling create-or-update-tags \
8  --tags \
9    ResourceId=<asg-name>,ResourceType=auto-scaling-group,Key=k8s.io/cluster-autoscaler/enabled,Value=true,PropagateAtLaunch=false \
10    ResourceId=<asg-name>,ResourceType=auto-scaling-group,Key=k8s.io/cluster-autoscaler/<cluster-name>,Value=owned,PropagateAtLaunch=false

If you used eksctl or Terraform to create the cluster, these tags may already be present. Verify:

aws autoscaling describe-tags \
  --filters Name=auto-scaling-group-name,Values=<asg-name> \
  --query "Tags[?Key=='k8s.io/cluster-autoscaler/enabled']"

Step 2: Create the IAM Policy

Cluster Autoscaler needs permission to describe and modify Auto Scaling Groups:

1cat > cluster-autoscaler-policy.json <<EOF
2{
3  "Version": "2012-10-17",
4  "Statement": [
5    {
6      "Effect": "Allow",
7      "Action": [
8        "autoscaling:DescribeAutoScalingGroups",
9        "autoscaling:DescribeAutoScalingInstances",
10        "autoscaling:DescribeLaunchConfigurations",
11        "autoscaling:DescribeScalingActivities",
12        "autoscaling:DescribeTags",
13        "autoscaling:SetDesiredCapacity",
14        "autoscaling:TerminateInstanceInAutoScalingGroup",
15        "ec2:DescribeImages",
16        "ec2:DescribeInstanceTypes",
17        "ec2:DescribeLaunchTemplateVersions",
18        "ec2:GetInstanceTypesFromInstanceRequirements",
19        "eks:DescribeNodegroup"
20      ],
21      "Resource": "*"
22    }
23  ]
24}
25EOF
26
27aws iam create-policy \
28  --policy-name ClusterAutoscalerPolicy \
29  --policy-document file://cluster-autoscaler-policy.json

Step 3: Create the IAM Role with IRSA

IRSA (IAM Roles for Service Accounts) lets the Cluster Autoscaler pod assume an IAM role without static credentials. This is the correct approach — never use node-level IAM permissions for this.

1# Get your OIDC provider URL
2OIDC_URL=$(aws eks describe-cluster \
3  --name my-cluster \
4  --query "cluster.identity.oidc.issuer" \
5  --output text | sed 's|https://||')
6
7ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
8POLICY_ARN="arn:aws:iam::${ACCOUNT_ID}:policy/ClusterAutoscalerPolicy"
9
10# Create the trust policy
11cat > trust-policy.json <<EOF
12{
13  "Version": "2012-10-17",
14  "Statement": [
15    {
16      "Effect": "Allow",
17      "Principal": {
18        "Federated": "arn:aws:iam::${ACCOUNT_ID}:oidc-provider/${OIDC_URL}"
19      },
20      "Action": "sts:AssumeRoleWithWebIdentity",
21      "Condition": {
22        "StringEquals": {
23          "${OIDC_URL}:sub": "system:serviceaccount:kube-system:cluster-autoscaler",
24          "${OIDC_URL}:aud": "sts.amazonaws.com"
25        }
26      }
27    }
28  ]
29}
30EOF
31
32# Create the role
33aws iam create-role \
34  --role-name ClusterAutoscalerRole \
35  --assume-role-policy-document file://trust-policy.json
36
37# Attach the policy
38aws iam attach-role-policy \
39  --role-name ClusterAutoscalerRole \
40  --policy-arn $POLICY_ARN
41
42ROLE_ARN=$(aws iam get-role \
43  --role-name ClusterAutoscalerRole \
44  --query "Role.Arn" --output text)
45echo "Role ARN: $ROLE_ARN"

Or with eksctl:

eksctl create iamserviceaccount \
  --cluster=my-cluster \
  --namespace=kube-system \
  --name=cluster-autoscaler \
  --attach-policy-arn=$POLICY_ARN \
  --approve

Step 4: Deploy Cluster Autoscaler via Helm

helm repo add autoscaler https://kubernetes.github.io/autoscaler
helm repo update

The Cluster Autoscaler version must match your Kubernetes minor version — CA 1.32.x for EKS 1.32 clusters, 1.31.x for 1.31, and so on. The Helm chart version maps to CA releases at kubernetes/autoscaler releases. Pin it explicitly; latest can land you on an incompatible version.

1helm install cluster-autoscaler autoscaler/cluster-autoscaler \
2  --namespace kube-system \
3  --version 9.46.0 \
4  --set autoDiscovery.clusterName=my-cluster \
5  --set awsRegion=ap-south-1 \
6  --set rbac.serviceAccount.create=true \
7  --set rbac.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=$ROLE_ARN \
8  --set extraArgs.balance-similar-node-groups=true \
9  --set extraArgs.skip-nodes-with-system-pods=false \
10  --set extraArgs.scale-down-delay-after-add=2m \
11  --set extraArgs.scale-down-unneeded-time=5m

Key flags explained:

• balance-similar-node-groups — distributes nodes evenly across AZs
• skip-nodes-with-system-pods=false — allows scale-down even if a node runs kube-proxy or DaemonSet pods
• scale-down-delay-after-add=2m — waits 2 minutes after a scale-up before evaluating scale-down (default is 10m — these are aggressive values suitable for dev/test; use 10m in production to avoid node flapping)
• scale-down-unneeded-time=5m — node must be underutilised for 5 minutes before being removed (default is 10m)

Step 5: Annotate the ServiceAccount (if created manually)

If you didn't use eksctl for IRSA, annotate the ServiceAccount:

kubectl annotate serviceaccount cluster-autoscaler \
  -n kube-system \
  eks.amazonaws.com/role-arn=$ROLE_ARN

Restart the deployment to pick up the annotation:

kubectl rollout restart deployment cluster-autoscaler -n kube-system

Step 6: Verify It's Working

# Check pod is running
kubectl get pods -n kube-system -l app.kubernetes.io/name=cluster-autoscaler

# Watch logs
kubectl logs -n kube-system -l app.kubernetes.io/name=cluster-autoscaler --tail=30 -f

Trigger a scale-up by creating a deployment that requests more resources than your current nodes have:

kubectl create deployment scale-test \
  --image=nginx \
  --replicas=50

kubectl get pods -w
# After a few minutes, pods will go from Pending to Running as new nodes join

Watch the node count increase:

kubectl get nodes -w

Step 7: Configure Pod Disruption Budgets for Safe Scale-Down

Cluster Autoscaler respects PodDisruptionBudgets when removing nodes. Set one on your critical deployments to prevent downtime during scale-down:

1kubectl apply -f - <<EOF
2apiVersion: policy/v1
3kind: PodDisruptionBudget
4metadata:
5  name: api-server-pdb
6spec:
7  minAvailable: 2     # At least 2 pods must remain available
8  selector:
9    matchLabels:
10      app: api-server
11EOF

Troubleshooting

Nodes not scaling up: Check logs for IAM permission errors. The most common issue is a missing OIDC provider or incorrect trust policy condition.

Nodes not scaling down: Check if pods have cluster-autoscaler.kubernetes.io/safe-to-evict: "false" annotation. Local storage or empty-dir volumes also block eviction.

Wrong node group being scaled: Ensure the ASG tags match exactly. The k8s.io/cluster-autoscaler/<cluster-name> tag must match your cluster name.

# Annotate pods that are safe to evict (override the default)
kubectl annotate pod <pod-name> cluster-autoscaler.kubernetes.io/safe-to-evict=true

Frequently Asked Questions

Why is Cluster Autoscaler not scaling up?

Work through it in order: pods must be pending and unschedulable for a resource reason, the node group's ASG must carry the discovery tags, the ASG must not already be at its maximum, and the IAM role must permit the autoscaling calls. The autoscaler's own logs state which of these it is blocked on, and reading them is faster than checking each in turn.

Why is it not scaling down?

Scale-down is deliberately conservative. A node is not removed if any pod on it has no controller, uses local storage, cannot be evicted under its PodDisruptionBudget, or is in kube-system without a budget allowing eviction. One such pod pins an entire node. The logs name the pod that is blocking removal.

Should I use Cluster Autoscaler or Karpenter on EKS?

Cluster Autoscaler if your workload shapes are uniform and you already run node groups you are happy with. Karpenter if shapes vary, you want faster provisioning, or you use spot heavily — it picks instance types itself instead of resizing predefined groups. Both are supported on EKS; Karpenter is the more actively developed path.

Do I need one node group per instance type?

With Cluster Autoscaler, effectively yes, because it can only add nodes to groups you have defined. That is why clusters with varied workloads accumulate node groups until the configuration becomes the problem. Keep groups to shapes you genuinely need, and treat sprawl as the signal to evaluate a provisioner that chooses instances itself.

Official References

• Cluster Autoscaler on AWS — Official setup guide for Cluster Autoscaler on EKS with IAM configuration
• Cluster Autoscaler FAQ — Comprehensive FAQ covering scale-up/down decisions, timing, and common issues
• EKS Managed Node Groups — AWS docs on managed node groups, the preferred target for Cluster Autoscaler on EKS
• EKS Karpenter — The modern alternative to Cluster Autoscaler: faster, more flexible, and AWS-native
• Amazon EKS Best Practices Guide — Cluster Autoscaler — AWS EKS team recommendations for autoscaling configuration