Kubernetes vs Nomad: A Deep, Practical Comparison

Kubernetes and Nomad are both answers to the same underlying question — "given a fleet of machines and a set of workloads, which machine runs which workload, and what happens when a machine dies?" — but they answer it from opposite ends. Kubernetes builds a rich, container-centric platform with dozens of built-in abstractions. Nomad builds a minimal, general-purpose scheduler and expects you to compose the rest from separate HashiCorp tools (Consul, Vault) or third-party pieces.

If you already know the one-paragraph verdict — Kubernetes is the default, Nomad is a deliberate, narrower choice — see our Docker Swarm vs Kubernetes vs Nomad comparison for that framing plus where Swarm fits (nowhere, in 2026). This post skips the survey and goes deep on the two orchestrators that are actually live contenders: what the same real deployment looks like in each, how their schedulers actually work, and what a migration between them costs in practice.

The core design split

Kubernetes' unit of work is the Pod, wrapped in higher-level controllers (Deployment, StatefulSet, DaemonSet, Job) that each encode a specific lifecycle policy. The API server persists desired state to etcd; a set of controllers continuously reconcile actual state toward it. This is a lot of moving parts, but each one is replaceable and inspectable — you can list every controller with kubectl api-resources and read its reconciliation logic.

Nomad's unit of work is the task, grouped into task groups, grouped into a job. There is no separate concept for "a thing that should always be running" versus "a thing that runs once and stops" versus "a thing that runs on every node" — a single job spec expresses all of these via a type field (service, batch, system, sysbatch). Nomad's server uses Raft for consensus (the same algorithm etcd uses, but embedded — no separate datastore to run). There's one binary, three roles (server, client, or both), and no separate API server process.

The practical consequence: a Kubernetes cluster has more independently-scaling, independently-failing components (API server, scheduler, controller-manager, etcd, kubelet, kube-proxy, CNI plugin, CoreDNS) than a Nomad cluster (server agents, client agents). That's real operational surface area, and it's the single biggest reason small teams find Nomad easier to run — not because Nomad does less, but because Nomad concentrates what it does into fewer processes.

The same deployment, two ways

To make the difference concrete, here's one real deployment — a stateless API behind a load balancer, 3 replicas, a rolling update policy, and a health check — expressed both ways.

Kubernetes (Deployment + Service, the minimum needed to actually run and reach this workload):

1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: api
5spec:
6  replicas: 3
7  strategy:
8    type: RollingUpdate
9    rollingUpdate:
10      maxSurge: 1
11      maxUnavailable: 0
12  selector:
13    matchLabels: { app: api }
14  template:
15    metadata:
16      labels: { app: api }
17    spec:
18      containers:
19        - name: api
20          image: myapp:v1.2.3
21          ports:
22            - containerPort: 8080
23          resources:
24            requests: { cpu: "500m", memory: "256Mi" }
25            limits: { memory: "256Mi" }
26          readinessProbe:
27            httpGet: { path: /healthz, port: 8080 }
28            periodSeconds: 5
29---
30apiVersion: v1
31kind: Service
32metadata:
33  name: api
34spec:
35  selector: { app: api }
36  ports:
37    - port: 80
38      targetPort: 8080

Nomad (a single job spec, HCL):

1job "api" {
2  datacenters = ["dc1"]
3  type        = "service"
4
5  update {
6    max_parallel = 1
7    canary       = 0
8  }
9
10  group "api" {
11    count = 3
12
13    network {
14      port "http" { to = 8080 }
15    }
16
17    service {
18      name = "api"
19      port = "http"
20      check {
21        type     = "http"
22        path     = "/healthz"
23        interval = "5s"
24        timeout  = "2s"
25      }
26    }
27
28    task "api" {
29      driver = "docker"
30      config {
31        image = "myapp:v1.2.3"
32        ports = ["http"]
33      }
34      resources {
35        cpu    = 500
36        memory = 256
37      }
38    }
39  }
40}

Two things to take from this side-by-side, not one:

1. Nomad's version is more concise for this exact case — one file, one block, no separate Service object, because Nomad's service stanza folds registration and health-checking into the job spec itself (backed by Consul if it's running, or Nomad's own built-in service discovery since Nomad 1.3+).
2. Kubernetes' version is more concise the moment you need what a Service actually buys you — stable ClusterIP, DNS name, and (with an Ingress or Gateway API resource) HTTP routing with path/header rules, TLS termination, and rate limiting, all as declarative objects with a huge ecosystem of controllers implementing them. Nomad's equivalent requires wiring in Consul Connect or an external load balancer explicitly; it's not missing, but it's not bundled either.

The "Nomad is simpler" claim is true for a narrow deployment and gets less true as the deployment grows real routing, real storage, and real multi-service dependencies — because that's exactly where Kubernetes' extra abstractions start paying for themselves.

Scheduler internals: bin-packing vs feasibility + ranking

Both schedulers solve a bin-packing problem — fit workloads onto nodes respecting resource requests, constraints, and affinity rules — but the algorithms differ in a way that matters at scale.

kube-scheduler runs a two-phase pipeline per Pod: filtering (which nodes can run this Pod at all — enough resources, taints tolerated, node selectors match) then scoring (rank the feasible nodes by plugins like NodeResourcesFit, InterPodAffinity, ImageLocality) and picks the highest-scoring node. This runs once per Pod, sequentially by default, though the scheduler parallelizes the filtering step across nodes internally. For very large clusters (thousands of nodes), this is the part that historically needed tuning (percentageOfNodesToScore) to stay fast, though modern kube-scheduler versions handle this adaptively.

Nomad's scheduler splits work into evaluations (a job change enqueues an evaluation) and plans (the scheduler's proposed placement). Critically, Nomad workers evaluate plans optimistically and in parallel — multiple scheduler workers can propose plans concurrently, and the Nomad server reconciles conflicts, retrying failed plans. This is a deliberate design choice for horizontal scheduler throughput: you add more scheduler workers (or servers) to increase placement throughput, rather than needing to optimize a single scheduling loop.

What this means in practice: for workloads with very high job/allocation churn — CI runners, batch processing pipelines spinning up and tearing down thousands of short-lived tasks per hour — Nomad's scheduler model has less inherent overhead per placement, because there's no equivalent of Kubernetes' Pod object lifecycle (creation, admission webhooks, kubelet sync loop, status updates all flowing back through the API server) for every allocation. This is precisely why Nomad has traction for CI/CD fleets and batch/HPC-style workloads even at organizations that run Kubernetes for their long-lived services.

Networking and service mesh

Kubernetes ships a networking model (every Pod gets a routable IP, Services provide stable virtual IPs, kube-proxy or an eBPF dataplane like Cilium implements it) but not a specific implementation — you choose a CNI plugin (Calico, Cilium, AWS VPC CNI, etc.). Service mesh is a separate layer entirely: Istio, Linkerd, or Cilium's own mesh mode, each adding a sidecar or eBPF-based dataplane for mTLS, retries, and traffic splitting. This is more moving parts, but also more choice — you pick the CNI and mesh that fit your actual requirements (eBPF for performance, Istio for the deepest traffic-management feature set).

Nomad has no networking model of its own beyond basic port allocation; it delegates service discovery and mesh capabilities entirely to Consul. Consul Connect provides mTLS and sidecar-based traffic management, configured via the connect stanza inside a Nomad job:

1service {
2  name = "api"
3  connect {
4    sidecar_service {
5      proxy {
6        upstreams {
7          destination_name = "database"
8          local_bind_port  = 5432
9        }
10      }
11    }
12  }
13}

This is genuinely elegant when you're already running Consul — service mesh becomes a few lines inside the job spec you're already writing, not a separate CRD-based system layered on top. But it also means Nomad's networking story is only as good as your Consul deployment; without Consul, you're back to manual port management and external load balancers. Kubernetes' Service abstraction, by contrast, works out of the box on every managed cluster with zero extra infrastructure.

Storage: CSI maturity gap

Both support the Container Storage Interface (CSI) standard — the same CSI plugins (the AWS EBS CSI driver, for example) can technically attach to either. In practice the maturity gap is real: Kubernetes' CSI ecosystem is the primary target for every storage vendor (Pure Storage, NetApp, Portworx, the cloud providers' own drivers), tested against Kubernetes' StatefulSet semantics — ordered pod startup/shutdown, stable per-replica identity, one PVC per replica that follows it across rescheduling. Nomad supports CSI volumes and has host volumes for simpler cases, but lacks a StatefulSet-equivalent primitive; running an ordered, stateful cluster (a database with a primary and replicas that must start in a specific order and keep stable identities) requires more manual work in the job spec, or relying on the application's own clustering logic rather than the orchestrator's.

If your workload is genuinely stateless or uses managed external state (RDS, a managed Kafka), this gap doesn't matter. If you're running self-managed stateful systems (PostgreSQL, Elasticsearch, Kafka) inside the cluster, Kubernetes' StatefulSet + CSI combination is meaningfully more mature.

Security model: RBAC + admission vs ACLs

Kubernetes' security surface is broad and layered: RBAC (Roles/ClusterRoles bound to subjects) controls API access; Pod Security Admission (the built-in replacement for the deprecated PodSecurityPolicy) enforces baseline/restricted pod-level constraints; NetworkPolicy objects (enforced by the CNI) restrict pod-to-pod traffic; admission webhooks (Kyverno, OPA/Gatekeeper) enforce arbitrary custom policy at the API layer before anything is persisted. This is a lot of surface to configure correctly, and misconfiguration here is a recurring real-world source of incidents — see our RBAC misconfigurations break production post for concrete failure patterns.

Nomad's access control is ACLs — a simpler token-and-policy model scoped to namespaces, with policies granting capabilities (submit-job, read-logs) or block-level access (a node { policy = "write" } block for node operations) rather than Kubernetes' verb-on-resource RBAC model. There's no direct Nomad equivalent to NetworkPolicy (that's Consul Connect intentions' job) or to admission webhooks — Sentinel, HashiCorp's policy-as-code framework, covers similar ground but is Nomad Enterprise-only, not available in the open-source tier. For teams that need fine-grained, auditable policy enforcement without paying for Enterprise, Kubernetes' open-source admission ecosystem (Kyverno, OPA/Gatekeeper) is currently the deeper toolset.

Autoscaling

Kubernetes has four autoscaling layers that compose: HPA (scale replica count on CPU/memory or custom metrics), VPA (right-size a single Pod's resource requests over time), KEDA (scale on external event sources — queue depth, Kafka lag, cron schedules — down to zero), and Karpenter (or the cluster autoscaler) provisioning and deprovisioning the underlying nodes to match. These are separate, independently-adopted projects that read the same Kubernetes API, which is both the strength (mix and match) and the complexity (four systems to understand).

Nomad has the Nomad Autoscaler, a single HashiCorp-maintained tool that handles both horizontal application scaling (via APM-source plugins — Prometheus, Datadog) and cluster (node count) scaling via cloud-provider plugins, driven by one policy block attached to the job:

1scaling {
2  min     = 2
3  max     = 10
4  policy {
5    check "cpu_usage" {
6      source = "prometheus"
7      query  = "avg(nomad_client_allocated_cpu)"
8      strategy "target-value" {
9        target = 70
10      }
11    }
12  }
13}

This is simpler to reason about — one tool, one policy syntax — but narrower: there's no direct Nomad equivalent to KEDA's 60+ event-source scalers, and VPA-style automatic resource-request tuning isn't a built-in capability. If your scaling triggers are CPU/memory or a metric already in Prometheus, Nomad Autoscaler covers it cleanly. If you need to scale a worker pool to zero based on SQS queue depth or scale a Pod's memory request based on its own historical usage, that's Kubernetes-ecosystem territory today.

Migrating between them: what actually has to change

Moving a real service from one to the other isn't a syntax translation — it's re-deriving the parts each platform gives you for free that the other doesn't.

Nomad → Kubernetes, the work is mostly addition: your job's Docker image and command line carry over directly, but you now write a Service (or Ingress/Gateway API resource) explicitly where Nomad's service stanza gave you registration for free, and you pick a CNI/mesh if you need Connect-equivalent mTLS. StatefulSet is new work if you were running anything stateful, since Nomad had no direct equivalent to translate from.

Kubernetes → Nomad, the work is mostly consolidation and loss: multiple Kubernetes objects (Deployment, Service, ConfigMap, Secret, NetworkPolicy) fold into fewer Nomad constructs, but you lose native StatefulSet ordering guarantees, lose the admission-webhook policy layer (Sentinel is the paid alternative), and lose direct access to any tool that only speaks the Kubernetes API — Argo CD, Istio, Kyverno, and most of the CNCF landscape have no Nomad equivalent to redeploy onto. This is the single largest practical blocker to Kubernetes → Nomad migrations: it's not that Nomad can't run the workload, it's that the surrounding platform tooling built up around a Kubernetes cluster usually can't move with it.

Who actually runs Nomad in production

Nomad's adoption is real but concentrated in a specific profile: organizations running very large, mixed-workload or batch-heavy fleets where Nomad's scheduler throughput and non-container task support are a direct fit. HashiCorp's own published case studies and conference talks (HashiConf) have documented Nomad running production workloads at scale at companies including Cloudflare (for parts of their edge compute and CI infrastructure) and Roblox (compute fleet orchestration), among others. The common thread across public Nomad adopters is scale combined with either heavy batch scheduling or a pre-existing HashiCorp stack (Vault + Consul) that made Nomad the path of least resistance — not teams choosing Nomad over Kubernetes for smaller, standard web-service workloads, where Kubernetes' ecosystem advantage dominates.

Decision framework

Don't decide this on philosophy — score your actual workload against the dimensions that matter:

If you score mixed and land in "either could work," default to Kubernetes — the ecosystem and hiring-pool advantages compound over the life of the platform in a way that's hard to unwind later. Choose Nomad when a specific row above is a real, current constraint, not a hypothetical future one.

Frequently Asked Questions

Does Nomad support anything like Kubernetes' Ingress or Gateway API?

Not natively. Nomad's job spec handles service registration and health checks, but HTTP-layer routing (path-based rules, header matching, TLS termination) is either delegated to Consul Connect's ingress gateway or handled by an external load balancer/API gateway you point at Nomad-registered services. It's not a gap that blocks anything, but it's explicitly separate infrastructure rather than a built-in Nomad object.

Can Nomad Autoscaler use KEDA-style event sources like queue depth?

Only indirectly — Nomad Autoscaler's APM plugins read from Prometheus, Datadog, and a few others, so if you export queue depth as a Prometheus metric, you can scale on it. There's no direct plugin ecosystem for the 60+ purpose-built event sources KEDA ships (SQS, Kafka lag, cron, etc.) — you'd wire the metric into Prometheus yourself first.

Is there a lightweight Nomad equivalent to k3s?

Not really needed the way k3s is needed for Kubernetes. Nomad's server and client agents are already single static binaries with minimal dependencies (no separate etcd, no control-plane component sprawl), so there isn't the same "full Kubernetes is too heavy" problem k3s exists to solve. A minimal Nomad cluster is already close to as light as Nomad gets.

Does the BSL license affect just running Nomad internally?

No. HashiCorp's Business Source License restricts building a competing commercial product on top of Nomad — offering it as a hosted orchestration service, for instance. Running Nomad to orchestrate your own company's workloads, internal or customer-facing, is unaffected. The licensing question only becomes real if you're a vendor considering embedding Nomad in something you sell.

Do any organizations run both Kubernetes and Nomad together?

Yes — the common pattern is Kubernetes for standard containerized services and Nomad for batch, CI, or mixed-workload fleets, sharing a Consul service mesh so services in either cluster can discover and call each other. It works, but it's a genuine dual-control-plane operational cost, not a shortcut — you're maintaining expertise and tooling for both.

For the three-way comparison including where Docker Swarm fits (nowhere, in 2026), see Docker Swarm vs Kubernetes vs Nomad. For managed Kubernetes platform choices once you've settled on Kubernetes, see EKS vs AKS: A Production Engineer's Comparison.

Weighing Kubernetes against Nomad for a real platform decision? Talk to us at Coding Protocols — we help teams make this call against their actual workload shape, not a generic comparison chart.

Official References

• Nomad: How the Scheduler Works — evaluations, plans, and worker parallelism
• Kubernetes Scheduling Framework — filter/score plugin architecture
• Nomad Autoscaler documentation — APM/target/strategy plugin model
• HashiCorp Business Source License FAQ — what BSL 1.1 does and doesn't restrict
• Consul Connect service mesh — sidecar proxy and mTLS model Nomad delegates to