Run GPU Workloads on EKS Auto Mode

Everything about running GPUs on Kubernetes gets tedious around the driver stack, not the scheduling. EKS Auto Mode's pitch is that it removes that part entirely: the NVIDIA drivers and device plugin ship inside AWS's AMI, so there's no GPU Operator to install and no DaemonSet to keep in sync with a kernel version. What it doesn't remove is the part everyone gets stuck on first — the built-in NodePools don't provision GPU capacity at all.

This tutorial builds the whole path: an Auto Mode cluster, a custom GPU NodePool, an nvidia-smi Job that proves the driver is real, and confirmation that the GPU node disappears on its own once nothing needs it. It's the hands-on companion to EKS Auto Mode for GPU Workloads — every claim below follows that post's reasoning about NodePools, node lifecycle, and cost.

What You'll Build

• An EKS cluster with Auto Mode enabled for compute, storage, and networking
• A custom NodePool that lets GPU pods actually get scheduled (the default general-purpose and system pools won't do this)
• An nvidia-smi Job that requests nvidia.com/gpu: 1 and proves the driver stack works with zero manual setup
• Verification that the GPU node scales back to zero automatically once the Job finishes

Step 1: Create an EKS Cluster With Auto Mode Enabled

Auto Mode is a cluster-level flag, not a separate product — you turn it on when you create the cluster and it takes over compute, storage, and networking automation. With eksctl:

1# cluster.yaml
2apiVersion: eksctl.io/v1alpha5
3kind: ClusterConfig
4
5metadata:
6  name: gpu-automode-demo
7  region: us-east-1
8  version: "1.31"
9
10autoModeConfig:
11  enabled: true
12  nodePools:
13    - general-purpose
14    - system

eksctl create cluster -f cluster.yaml

nodePools here just enables or disables AWS's two built-in pools — you cannot edit what's inside them. That's the detail this tutorial exists to work around: neither pool selects an accelerated instance type, so a GPU pod scheduled onto a fresh Auto Mode cluster sits Pending indefinitely. (If you're provisioning via the AWS CLI or Terraform instead of eksctl, the equivalent is aws eks create-cluster --compute-config enabled=true ... — the flag exists, but eksctl is the more concrete example to follow through this tutorial.)

Confirm the cluster is up and kubectl reaches it:

kubectl get nodes
# expect a couple of general-purpose nodes; no GPU node yet — nothing has requested one

Step 2: Add a GPU NodePool

You still have to author a NodePool — that part isn't automated away. What Auto Mode removes is everything underneath it: you don't write a NodeClass, you don't run the Karpenter controller yourself, and you don't ship a device-plugin DaemonSet or install NVIDIA drivers. Auto Mode's own Karpenter installation reads your NodePool, and its AMI already has the driver stack baked in.

1# gpu-nodepool.yaml
2apiVersion: karpenter.sh/v1
3kind: NodePool
4metadata:
5  name: gpu-demo
6spec:
7  template:
8    metadata:
9      labels:
10        workload: gpu
11    spec:
12      nodeClassRef:
13        group: eks.amazonaws.com
14        kind: NodeClass
15        name: default
16
17      requirements:
18        - key: "eks.amazonaws.com/instance-family"
19          operator: In
20          values: ["g5", "g6"]
21        - key: "eks.amazonaws.com/instance-gpu-manufacturer"
22          operator: In
23          values: ["nvidia"]
24        - key: "karpenter.sh/capacity-type"
25          operator: In
26          values: ["on-demand"]
27
28      taints:
29        - key: nvidia.com/gpu
30          effect: NoSchedule
31
32  disruption:
33    consolidationPolicy: WhenEmpty
34    consolidateAfter: 1m
35
36  limits:
37    cpu: "64"

kubectl apply -f gpu-nodepool.yaml

Two choices worth calling out:

• nodeClassRef points at default — the built-in NodeClass Auto Mode ships with. You're not authoring AMI selection, subnet discovery, or security groups; that's exactly what stays automatic.
• consolidationPolicy: WhenEmpty instead of the general-purpose default of WhenEmptyOrUnderutilized. GPU nodes shouldn't get repacked mid-job — you want them reclaimed only when genuinely idle, never shuffled while something is using the GPU. consolidateAfter: 1m is shortened from the usual 5m purely so this demo doesn't sit around; leave it longer in production if node churn on brief idle gaps is a concern.

Step 3: Deploy a GPU-Requesting Job

nvidia-smi in a container is enough to prove the whole path works — it doesn't need to be a real training or inference workload.

1# nvidia-smi-job.yaml
2apiVersion: batch/v1
3kind: Job
4metadata:
5  name: gpu-smoke-test
6  namespace: default
7spec:
8  backoffLimit: 0
9  template:
10    spec:
11      restartPolicy: Never
12      nodeSelector:
13        workload: gpu
14      tolerations:
15        - key: nvidia.com/gpu
16          operator: Exists
17          effect: NoSchedule
18      containers:
19        - name: nvidia-smi
20          image: nvidia/cuda:12.4.1-base-ubuntu22.04
21          command: ["nvidia-smi"]
22          resources:
23            limits:
24              nvidia.com/gpu: 1

kubectl apply -f nvidia-smi-job.yaml

The nodeSelector and toleration are what actually connect this pod to the NodePool from Step 2 — the taint on the NodePool keeps ordinary workloads off expensive GPU hardware, and the pod has to explicitly opt in. Nothing else about requesting a GPU differs from any other Kubernetes cluster: resources.limits.nvidia.com/gpu is the standard device-plugin API, Auto Mode just guarantees something is listening on the other end of it.

Step 4: Watch Auto Mode Provision the Node

kubectl get pods -w

The pod starts Pending — there's no GPU node yet. Watch Karpenter respond:

kubectl get nodeclaims -w

A new NodeClaim appears almost immediately, matching the gpu-demo NodePool's requirements. Auto Mode picks an instance size from the g5/g6 family on its own — you didn't specify an exact type, only the family and the GPU requirement. Within a minute or two the claim resolves to a running node and the pod moves to Running:

kubectl get nodes -l workload=gpu -o wide

No AMI to pick, no launch template, no manual Karpenter NodePool for driver bootstrapping — the only YAML you wrote was the NodePool in Step 2, and that was about GPU family selection, not driver installation.

Step 5: Verify the GPU Is Actually Usable

A node with a GPU attached isn't the same as a container that can see it. Check the Job's output:

kubectl logs job/gpu-smoke-test

You should see standard nvidia-smi table output, something like:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 550.xx       Driver Version: 550.xx       CUDA Version: 12.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  NVIDIA A10G          Off | 00000000:00:1E.0 Off |                    0 |
| 0%   28C    P0             45W / 300W|      0MiB / 23028MiB |      0%   Default |
+-----------------------------------------------------------------------------+

The exact GPU model and driver version depend on which instance Auto Mode picked from the g5/g6 family, but the shape is the same: a real driver, a real device, visible inside the container, with nothing installed by you beyond the container image itself. If this comes back empty or the Job errors instead, check kubectl describe pod first — a missing toleration or a NodePool that didn't match any capacity are the two most common causes at this stage, not a driver problem.

Step 6: Confirm the GPU Node Scales Back to Zero

Once the Job completes, nothing else is requesting workload: gpu. With consolidationPolicy: WhenEmpty, Auto Mode reclaims the node on its own:

kubectl get nodeclaims -w
# watch the gpu-demo NodeClaim disappear after consolidateAfter elapses

kubectl get nodes -l workload=gpu
# should return no resources once consolidation completes

This is the part that matters for the bill. GPU instances are billed by the second while running, whether or not anything is using them — a static node group holding a g5 node "just in case" burns money every hour it sits idle. WhenEmpty consolidation means the expensive hardware only exists while something needs it, which is the same point the blog post makes about utilization being what actually drives GPU cost, not the per-node management fee.

What Auto Mode Doesn't Give You

Everything above is the part that gets easier. Three constraints don't go away:

• No SSH, no SSM, and you can't install anything on the node. If your GPU debugging habit starts with logging into the box, it has to become kubectl debug and pod-level tooling instead.
• You can't pin a driver version. AWS rotates the AMI roughly weekly for CVE and security fixes, and your node goes with it — nodes also expire after 336 hours (14 days) by default, with a hard 21-day maximum lifetime regardless of what you configure. For the short-lived Job in this tutorial that's irrelevant; for a multi-week training run that can't checkpoint, it's a real hazard.
• IMDSv2 has a hop limit of 1, non-configurable. ML tooling that reaches for instance metadata to discover region or credentials needs hostNetwork: true to work at all — prefer EKS Pod Identity and sidestep it.

If any of those are disqualifying — you need a pinned CUDA version, host-level profiling access, or a training job that genuinely can't tolerate a node disappearing on AWS's schedule rather than yours — that's the signal to run self-managed Karpenter for that workload instead. Auto Mode and self-managed Karpenter aren't mutually exclusive within one cluster: AWS labels every Auto Mode node eks.amazonaws.com/compute-type: auto, so you can run inference on Auto Mode and training on a hand-rolled NodePool side by side.

Cost and Teardown

Everything provisioned in this tutorial bills by the second while it exists. Delete the workload first, then the resources it depends on:

kubectl delete job gpu-smoke-test
kubectl delete -f gpu-nodepool.yaml

Confirm no GPU node is still running before you walk away:

kubectl get nodes -l workload=gpu
# should be empty

If you created gpu-automode-demo solely for this tutorial, delete the whole cluster rather than leaving it idle — the control plane and any lingering general-purpose nodes still cost money even with zero GPU nodes attached:

eksctl delete cluster -f cluster.yaml

Where to Go Next

• Read the full reasoning behind this build — EKS Auto Mode for GPU Workloads covers the NodePool labels, the node lifecycle trade-offs, and the cost framing this tutorial is built on.
• Compare the managed-GPU model across clouds — EKS Auto Mode vs GKE Autopilot covers how the two platforms differ beyond just GPU support.
• When Auto Mode's node lifecycle is disqualifying — How to Install Karpenter on EKS is the manual alternative where you own the AMI and the disruption schedule.
• GPU scheduling beyond EKS specifics — Kubernetes GPU and Machine Learning Workloads covers the broader scheduling patterns that apply whether or not you're on Auto Mode.