Multi-Cluster GitOps with ArgoCD ApplicationSets
Quick answer
Generate an Application per cluster and per Git directory instead of hand-writing one, using ArgoCD's Cluster and Git generators — then stage the rollout across clusters with RollingSync.
- Step 1: Register a Second Cluster
- Step 2: The Cluster Generator
- Step 3: The Git Generator (Directory Mode)
- Step 4: The List Generator — the Manual Alternative
- Step 5: Staged Rollouts with RollingSync
advanced · 16 min
Before you begin
- ArgoCD already installed (see the Install ArgoCD tutorial)
- Two or more clusters you can register (kind clusters are fine for practicing)
- kubectl and argocd CLI
- A Git repo you can push manifests to
If readers new to ArgoCD's Application concept haven't already, read ArgoCD App-of-Apps: Managing Multi-Environment Clusters first — this tutorial assumes that foundation and picks up exactly where it stops being enough.
App-of-apps solves nesting: one parent Application whose children are other Application manifests, so you don't run argocd app create by hand for every service. But every child in that pattern is still a manifest someone wrote. Roll the same Helm chart out to five clusters with app-of-apps and you write five Application YAMLs — one per cluster — that someone has to keep in sync as clusters are added, renamed, or decommissioned. Add a sixth cluster and nothing in Git tells you an Application is missing until someone notices the workload isn't running there.
ApplicationSets replace the manifest with a template plus a generator. You write one ApplicationSet, point its generator at a list of clusters (or a set of Git directories), and ArgoCD produces the Application resources itself — for every cluster currently registered, or every directory currently in the repo, no manual step required.
What You'll Build
- A second cluster registered with ArgoCD, so you have real multi-cluster fan-out to test
- An ApplicationSet using the Cluster generator that deploys the same chart to every registered cluster automatically
- An ApplicationSet using the Git generator (directory mode) that generates one Application per environment directory in a monorepo
- A look at the List generator as the manual alternative, and when it's still the right call
- A
RollingSyncstrategy that stages a rollout across cluster groups instead of hitting every cluster at once
Step 1: Register a Second Cluster
ApplicationSets only fan out across clusters ArgoCD already knows about. If you're only registered against in-cluster, add a second target:
kind create cluster --name workload-2
# Get the context name and register it with ArgoCD
kubectl config get-contexts
argocd cluster add kind-workload-2 --name workload-2Confirm both clusters are visible to ArgoCD:
argocd cluster list
# SERVER NAME VERSION STATUS
# https://kubernetes.default.svc in-cluster Successful
# https://<workload-2-api> workload-2 1.29 Successfulargocd cluster add writes a Secret labeled argocd.argoproj.io/secret-type: cluster into the argocd namespace. That label is exactly what the Cluster generator queries — nothing more is required to make a cluster a valid ApplicationSet target.
Step 2: The Cluster Generator
Create appset-cluster.yaml. This ApplicationSet deploys charts/nginx to every cluster registered with ArgoCD, present or future:
1# appset-cluster.yaml
2apiVersion: argoproj.io/v1alpha1
3kind: ApplicationSet
4metadata:
5 name: nginx-all-clusters
6 namespace: argocd
7spec:
8 goTemplate: true
9 goTemplateOptions: ["missingkey=error"]
10 generators:
11 - clusters: {}
12 template:
13 metadata:
14 name: "nginx-{{.name}}"
15 spec:
16 project: default
17 source:
18 repoURL: https://github.com/your-org/gitops-repo
19 targetRevision: main
20 path: charts/nginx
21 helm:
22 valueFiles:
23 - values-base.yaml
24 destination:
25 server: "{{.server}}"
26 namespace: nginx
27 syncPolicy:
28 automated:
29 prune: true
30 selfHeal: true
31 syncOptions:
32 - CreateNamespace=truekubectl apply -f appset-cluster.yamlgenerators: [{ clusters: {} }] with no selector matches every cluster secret in argocd, including the implicit in-cluster entry. {{.name}} and {{.server}} are fields the generator emits per cluster — .name is what you passed to --name when registering, .server is the API endpoint. The template runs once per cluster, so two registered clusters produce two Applications: nginx-in-cluster and nginx-workload-2, each pointed at its own destination.server.
To target a subset instead of every cluster, add labels when registering and filter with a selector:
argocd cluster add kind-workload-2 --name workload-2 --label env=staginggenerators:
- clusters:
selector:
matchLabels:
env: stagingRegister a third cluster tomorrow and this ApplicationSet needs zero changes — the next reconcile loop picks up the new cluster secret and generates its Application automatically.
Step 3: The Git Generator (Directory Mode)
Cluster generators solve "same app, many clusters." Git generators solve a different shape: "one Application per directory," which is the natural fit for a monorepo with environments/dev, environments/staging, environments/prod.
gitops-repo/
└── environments/
├── dev/
│ └── kustomization.yaml
├── staging/
│ └── kustomization.yaml
└── prod/
└── kustomization.yaml
1# appset-git-directories.yaml
2apiVersion: argoproj.io/v1alpha1
3kind: ApplicationSet
4metadata:
5 name: nginx-by-environment
6 namespace: argocd
7spec:
8 goTemplate: true
9 goTemplateOptions: ["missingkey=error"]
10 generators:
11 - git:
12 repoURL: https://github.com/your-org/gitops-repo
13 revision: main
14 directories:
15 - path: environments/*
16 template:
17 metadata:
18 name: "nginx-{{.path.basename}}"
19 spec:
20 project: default
21 source:
22 repoURL: https://github.com/your-org/gitops-repo
23 targetRevision: main
24 path: "{{.path.path}}"
25 destination:
26 server: https://kubernetes.default.svc
27 namespace: "{{.path.basename}}"
28 syncPolicy:
29 automated:
30 prune: true
31 selfHeal: true
32 syncOptions:
33 - CreateNamespace=truekubectl apply -f appset-git-directories.yamldirectories.path: environments/* matches every immediate subdirectory of environments/. {{.path.path}} and {{.path.basename}} are generator outputs — {{.path.path}} is the full matched path (environments/prod), {{.path.basename}} is the last segment (prod). This uses the same goTemplate: true dotted syntax as the Cluster generator in Step 2 — current ArgoCD docs show every generator this way now, so it's worth being consistent about it across ApplicationSets rather than mixing template styles. Add environments/canary/ to the repo and push — no ArgoCD config change, no kubectl apply — and the next reconcile creates nginx-canary on its own.
Git generators also support file mode, matching a glob of config files (e.g. environments/*/config.json) instead of directories, useful when each environment's parameters live in a single file rather than a directory of manifests. Directory mode is the more common fit for GitOps repos structured like the one above.
Step 4: The List Generator — the Manual Alternative
Not every rollout needs auto-discovery. If you have three fixed regions that almost never change and want the target list to live directly in the ApplicationSet — reviewable in one PR diff, no dependency on cluster registration or repo layout — use a List generator:
1spec:
2 goTemplate: true
3 generators:
4 - list:
5 elements:
6 - cluster: us-east
7 url: https://us-east.k8s.internal
8 - cluster: eu-west
9 url: https://eu-west.k8s.internal
10 - cluster: ap-south
11 url: https://ap-south.k8s.internalWith goTemplate: true set (as in every generator in this tutorial), template fields reference {{.cluster}} and {{.url}} the same way Cluster generator fields reference {{.name}}/{{.server}}. Drop that line and the dotted syntax stops resolving — goTemplate isn't optional once you're using this syntax anywhere in the spec. Reach for List when the set of targets is small, static, and you'd rather the diff show up as an explicit list edit than as an implicit consequence of registering a cluster or adding a directory elsewhere. Reach for Cluster or Git generators once that list is large enough, or changes often enough, that maintaining it by hand is the same manual-per-target problem ApplicationSets exist to remove.
Step 5: Staged Rollouts with RollingSync
syncPolicy.automated on each generated Application means every cluster syncs the moment the source changes — same blast radius as looping kubectl apply across clusters, just automated. For multi-cluster, that's usually not what you want: a bad chart change syncing to all N clusters simultaneously is the incident, not the deploy.
strategy.type: RollingSync staggers the sync across cluster groups, waiting for each group to report healthy before advancing to the next — by taking over sync scheduling itself, not by layering on top of automated. The two are mutually exclusive: if a template still sets syncPolicy.automated, ArgoCD forces autosync off anyway once RollingSync is active and logs a warning in the applicationset-controller. So the automated block from Step 2 needs to come out, not stay in.
matchExpressions in each step matches the generated Application's own labels — and those aren't inherited from the cluster secret automatically. The template has to carry the label through explicitly:
1# appset-cluster.yaml — automated removed, labels + strategy added
2spec:
3 goTemplate: true
4 goTemplateOptions: ["missingkey=error"]
5 generators:
6 - clusters: {}
7 template:
8 metadata:
9 name: "nginx-{{.name}}"
10 labels:
11 env: '{{index .metadata.labels "env"}}'
12 spec:
13 project: default
14 source:
15 repoURL: https://github.com/your-org/gitops-repo
16 targetRevision: main
17 path: charts/nginx
18 helm:
19 valueFiles:
20 - values-base.yaml
21 destination:
22 server: "{{.server}}"
23 namespace: nginx
24 syncPolicy:
25 syncOptions:
26 - CreateNamespace=true
27 strategy:
28 type: RollingSync
29 rollingSync:
30 steps:
31 - matchExpressions:
32 - key: env
33 operator: In
34 values: [staging]
35 maxUpdate: 50%
36 - matchExpressions:
37 - key: env
38 operator: In
39 values: [prod]
40 maxUpdate: 25%{{index .metadata.labels "env"}} reads the env label straight off the cluster secret and copies it onto the generated Application, which is what makes it visible to matchExpressions at all. Any cluster without that label renders an empty string, and an Application labelled env: "" matches neither step — upstream's rule is that "if an Application is not selected in any step, it will be excluded from the rolling sync and needs to be manually synced through the CLI or UI." The implicit in-cluster entry is the one that bites here, since it has no env label and nothing prompts you to give it one: nginx-in-cluster will silently sit outside the rollout. Either label it like any other target, or exclude it from the generator with a selector:
generators:
- clusters:
selector:
matchExpressions:
- key: env
operator: ExistsThat means every cluster you want included in a step needs the label set on its secret first — the same --label flag from Step 1 works for clusters you're registering fresh (argocd cluster add kind-workload-3 --name workload-3 --label env=prod); for a cluster already registered, re-running argocd cluster add against it is safe and just updates the existing secret.
ArgoCD syncs step 1 fully, waits for every matched Application in that step to reach Synced/Healthy, then proceeds to step 2 — with automated gone, "sync" here means the RollingSync controller triggers each Application's sync operation directly as its turn comes up, not that the Application free-runs on every Git change outside the staged rollout. maxUpdate caps how many Applications within a step sync concurrently — 25% on the prod step means a bad rollout stops after roughly a quarter of prod clusters instead of all of them.
One more thing before this does anything: Progressive Syncs (the RollingSync strategy) ships as a Beta feature that's off by default. Enable it on the applicationset-controller with --enable-progressive-syncs, or set ARGOCD_APPLICATIONSET_CONTROLLER_ENABLE_PROGRESSIVE_SYNCS=true (or the equivalent key in the argocd-cmd-params-cm ConfigMap) — without that, the strategy block above is silently ignored and every generated Application just sits unsynced, since automated is gone and nothing else is triggering a sync.
This is the feature that makes ApplicationSets more than a for-loop around Applications: app-of-apps has no equivalent staged-rollout primitive, because its children are separate manifests with no shared coordination.
Step 6: Verify
Confirm the Cluster generator produced one Application per registered cluster:
1argocd appset list
2# NAME PROJECT
3# nginx-all-clusters default
4# nginx-by-environment default
5
6argocd app list
7# NAME CLUSTER NAMESPACE STATUS HEALTH
8# nginx-in-cluster in-cluster nginx Synced Healthy
9# nginx-workload-2 workload-2 nginx Synced Healthy
10# nginx-dev in-cluster dev Synced Healthy
11# nginx-staging in-cluster staging Synced Healthy
12# nginx-prod in-cluster prod Synced HealthyFive Applications, zero of which you wrote by hand. Now prove the Git generator actually regenerates on repo changes — add a new environment directory and push:
mkdir -p environments/qa
cat <<'EOF' > environments/qa/kustomization.yaml
resources:
- ../../base
EOF
git add environments/qa && git commit -m "Add qa environment" && git push# wait for the next reconcile (default requeue is ~3 min), then check:
argocd appset get nginx-by-environment
argocd app list
# ... previous rows ...
# nginx-qa in-cluster qa Synced Healthynginx-qa appears without touching ArgoCD directly — that's the entire point. If you deregister workload-2 (argocd cluster rm workload-2), the Cluster generator's nginx-workload-2 Application is removed the same way, since the generator's output is always the current cluster list, not a snapshot from when you first applied the ApplicationSet.
Where to Go Next
- ArgoCD App-of-Apps: Managing Multi-Environment Clusters — the manual/nested pattern this tutorial builds on top of
- ArgoCD Setup and Automated Sync — sync policy fields (
automated,prune,selfHeal) in more depth - ArgoCD GitOps: Getting Started — fundamentals if Application, sync, and repo structure are still new
- Install ArgoCD — installation, if you're starting from zero
We built Podscape to simplify Kubernetes workflows like this — logs, events, and cluster state in one interface, without switching tools.
Struggling with this in production?
We help teams fix these exact issues. Our engineers have deployed these patterns across production environments at scale.