Kubernetes Operators: Building Custom Controllers with CRDs (2026)

Kubernetes ships with built-in controllers for Deployments, StatefulSets, Services, and other core resource types. Each controller watches its resource type, compares desired state (the spec) against actual state (the status), and takes actions to reconcile them. An operator is this same pattern applied to your own resource types — you define the CRD, you write the controller logic.

The classic example: a PostgresCluster CRD managed by an operator that knows how to provision a Patroni HA cluster, run backups, rotate credentials, perform failovers, and upgrade the database version — all triggered by changes to the PostgresCluster resource, rather than requiring manual operational procedures.

When to Build an Operator

Before writing a controller, assess whether you actually need one:

Good candidates for operators:

Stateful infrastructure with complex lifecycle management (database clusters, message brokers, certificate authorities)
Multi-resource abstractions that always go together (Deployment + Service + HPA + PDB + ServiceMonitor = "Application")
Day-2 operations that are currently manual and error-prone (backup schedules, version upgrades, certificate rotation)
Platform primitives you want to expose as Kubernetes-native self-service (see the Internal Developer Platform pattern with Crossplane)

Cases where an operator is overkill:

You need a one-time setup task → use a Job or Helm post-install hook
You need to sync config from an external source → use External Secrets Operator or Flux
You need to provision cloud infrastructure → use Crossplane (which is itself an operator framework)
The operational logic can be expressed in a Helm chart with hooks → use Helm

Operators add operational overhead: they need to be deployed, monitored, upgraded, and debugged. The value must exceed that cost.

CRD Design

A Custom Resource Definition registers a new resource type with the Kubernetes API:

yaml

1apiVersion: apiextensions.k8s.io/v1
2kind: CustomResourceDefinition
3metadata:
4  name: postgresclusters.db.example.com
5spec:
6  group: db.example.com
7  names:
8    kind: PostgresCluster
9    plural: postgresclusters
10    singular: postgrescluster
11    shortNames: ["pgc"]
12  scope: Namespaced
13  versions:
14    - name: v1alpha1
15      served: true
16      storage: true
17      schema:
18        openAPIV3Schema:
19          type: object
20          properties:
21            spec:
22              type: object
23              required: ["version", "replicas"]
24              properties:
25                version:
26                  type: string
27                  enum: ["15", "16"]
28                  description: "PostgreSQL major version"
29                replicas:
30                  type: integer
31                  minimum: 1
32                  maximum: 9
33                  description: "Number of PostgreSQL replicas"
34                storageGB:
35                  type: integer
36                  minimum: 10
37                  default: 50
38                  description: "Storage size in GB per replica"
39                backupSchedule:
40                  type: string
41                  description: "Cron expression for backup schedule"
42            status:
43              type: object
44              properties:
45                phase:
46                  type: string
47                  enum: ["Pending", "Running", "Degraded", "Failed"]
48                readyReplicas:
49                  type: integer
50                primaryPod:
51                  type: string
52                conditions:
53                  type: array
54                  x-kubernetes-list-type: map
55                  x-kubernetes-list-map-keys: [type]
56                  items:
57                    type: object
58                    required: [type, status]
59                    properties:
60                      type:
61                        type: string
62                      status:
63                        type: string
64                      reason:
65                        type: string
66                      message:
67                        type: string
68                      lastTransitionTime:
69                        type: string
70                        format: date-time
71                      observedGeneration:
72                        type: integer
73      subresources:
74        status: {}   # Enable /status subresource for controller to update status separately
75      additionalPrinterColumns:
76        - name: Version
77          type: string
78          jsonPath: .spec.version
79        - name: Replicas
80          type: integer
81          jsonPath: .spec.replicas
82        - name: Phase
83          type: string
84          jsonPath: .status.phase
85        - name: Age
86          type: date
87          jsonPath: .metadata.creationTimestamp

After applying this CRD, users can create PostgresCluster resources:

yaml

1apiVersion: db.example.com/v1alpha1
2kind: PostgresCluster
3metadata:
4  name: payments-db
5  namespace: production
6spec:
7  version: "16"
8  replicas: 3
9  storageGB: 100
10  backupSchedule: "0 2 * * *"

And query them like any native resource:

bash

kubectl get pgc -n production
# NAME          VERSION   REPLICAS   PHASE     AGE
# payments-db   16        3          Running   2d

Schema Design Principles

Use required for fields that must always be present. Optional fields with sensible defaults are better than required fields with arbitrary values — but don't default critical security/operational choices.

Put all user-controlled fields in spec. The status subresource is written by the controller, not the user. Enabling the /status subresource ensures users can't modify status directly — only the controller can.

Version your API from the start. Start with v1alpha1. Promote to v1beta1 when the API stabilises. Graduate to v1 when you're committed to backwards compatibility. API versioning in Kubernetes is immutable — you can't change field types or remove required fields in a version.

Controller Architecture

A controller watches resources and reconciles desired state with actual state. The reconcile loop:

Watch (CRD events) → Queue → Reconcile → Update Status
        ↑                                      |
        └──────────── Watch (dependent resources) ←─┘

Using controller-runtime

controller-runtime is the standard library for writing Kubernetes controllers in Go. kubebuilder scaffolds controller-runtime projects with generated CRD types, RBAC annotations, and test infrastructure.

bash

1# Install kubebuilder — use curl from GitHub releases for the latest version
2# Note: Homebrew formula may lag behind GitHub releases. Use the curl method for the latest version.
3KUBEBUILDER_VERSION=$(curl -s https://api.github.com/repos/kubernetes-sigs/kubebuilder/releases/latest | jq -r .tag_name)
4curl -L -o kubebuilder "https://github.com/kubernetes-sigs/kubebuilder/releases/download/${KUBEBUILDER_VERSION}/kubebuilder_$(go env GOOS)_$(go env GOARCH)"
5chmod +x kubebuilder && mv kubebuilder /usr/local/bin/
6
7# Scaffold a new operator project
8mkdir postgres-operator && cd postgres-operator
9kubebuilder init --domain example.com --repo github.com/my-org/postgres-operator
10
11# Scaffold the API and controller
12kubebuilder create api \
13  --group db \
14  --version v1alpha1 \
15  --kind PostgresCluster

This generates:

api/v1alpha1/postgrescluster_types.go — Go struct matching the CRD schema
internal/controller/postgrescluster_controller.go — the reconcile loop
config/crd/bases/ — generated CRD YAML from Go struct annotations
config/rbac/ — RBAC rules generated from controller annotations

The Reconcile Loop

1// internal/controller/postgrescluster_controller.go
2
3func (r *PostgresClusterReconciler) Reconcile(
4    ctx context.Context,
5    req ctrl.Request,
6) (ctrl.Result, error) {
7    log := log.FromContext(ctx)
8
9    // Fetch the PostgresCluster resource
10    cluster := &dbv1alpha1.PostgresCluster{}
11    if err := r.Get(ctx, req.NamespacedName, cluster); err != nil {
12        // Resource was deleted — clean up if necessary
13        return ctrl.Result{}, client.IgnoreNotFound(err)
14    }
15
16    // Set a finalizer on the first reconcile
17    if !controllerutil.ContainsFinalizer(cluster, finalizerName) {
18        controllerutil.AddFinalizer(cluster, finalizerName)
19        if err := r.Update(ctx, cluster); err != nil {
20            return ctrl.Result{}, err
21        }
22    }
23
24    // Handle deletion
25    if !cluster.DeletionTimestamp.IsZero() {
26        return r.reconcileDelete(ctx, cluster)
27    }
28
29    // Main reconcile logic
30    if err := r.reconcileStatefulSet(ctx, cluster); err != nil {
31        // "sigs.k8s.io/controller-runtime/pkg/controller/controllerutil" for meta.SetStatusCondition
32        meta.SetStatusCondition(&cluster.Status.Conditions, metav1.Condition{
33            Type:               "StatefulSetReady",
34            Status:             metav1.ConditionFalse,
35            Reason:             "ReconcileError",
36            Message:            err.Error(),
37            LastTransitionTime: metav1.Now(),
38        })
39        return ctrl.Result{}, err
40    }
41
42    if err := r.reconcileService(ctx, cluster); err != nil {
43        return ctrl.Result{}, err
44    }
45
46    if err := r.reconcileBackupSchedule(ctx, cluster); err != nil {
47        return ctrl.Result{}, err
48    }
49
50    // Update status
51    cluster.Status.Phase = "Running"
52    cluster.Status.ReadyReplicas = int32(r.getReadyReplicas(ctx, cluster))
53    if err := r.Status().Update(ctx, cluster); err != nil {
54        return ctrl.Result{}, err
55    }
56
57    log.Info("Reconciliation complete", "cluster", cluster.Name, "phase", cluster.Status.Phase)
58    return ctrl.Result{RequeueAfter: time.Minute * 5}, nil  // Requeue every 5 minutes
59}

Key patterns:

Idempotent reconcile: The reconcile function is called any time the resource or a watched dependent changes. It must produce the same result whether called once or ten times.
Return errors to requeue: Returning an error from Reconcile puts the resource back in the queue with exponential backoff. Don't return errors for expected conditions — update status and return ctrl.Result{} (success).
Use RequeueAfter for periodic checks: Resources with external state (a database that might become unhealthy between events) should requeue periodically.

Watching Dependent Resources

Your controller should reconcile when owned resources change, not just when the CRD is updated:

1func (r *PostgresClusterReconciler) SetupWithManager(mgr ctrl.Manager) error {
2    return ctrl.NewControllerManagedBy(mgr).
3        For(&dbv1alpha1.PostgresCluster{}).   // Watch the CRD
4        Owns(&appsv1.StatefulSet{}).           // Requeue when owned StatefulSets change
5        Owns(&corev1.Service{}).               // Requeue when owned Services change
6        Owns(&batchv1.CronJob{}).              // Requeue when owned CronJobs change
7        Complete(r)
8}

Owns watches for changes to resources where the owner reference points to the PostgresCluster. When a StatefulSet replica count changes (e.g., due to HPA or manual edit), the PostgresCluster controller is triggered to reconcile.

Owner References

Set owner references on all resources your operator creates so they're garbage collected when the CRD is deleted:

1func (r *PostgresClusterReconciler) reconcileStatefulSet(
2    ctx context.Context,
3    cluster *dbv1alpha1.PostgresCluster,
4) error {
5    sts := &appsv1.StatefulSet{
6        ObjectMeta: metav1.ObjectMeta{
7            Name:      cluster.Name,
8            Namespace: cluster.Namespace,
9        },
10    }
11
12    _, err := controllerutil.CreateOrUpdate(ctx, r.Client, sts, func() error {
13        // Set owner reference — garbage collected with the PostgresCluster
14        if err := controllerutil.SetControllerReference(cluster, sts, r.Scheme); err != nil {
15            return err
16        }
17        // Set the StatefulSet spec
18        sts.Spec = buildStatefulSetSpec(cluster)
19        return nil
20    })
21    return err
22}

CreateOrUpdate is idempotent — it creates the resource if it doesn't exist, or updates it if it does. The mutate function is called in both cases to set the desired state.

RBAC for Operators

Generate RBAC rules from annotations on the controller struct:

1//+kubebuilder:rbac:groups=db.example.com,resources=postgresclusters,verbs=get;list;watch;create;update;patch;delete
2//+kubebuilder:rbac:groups=db.example.com,resources=postgresclusters/status,verbs=get;update;patch
3//+kubebuilder:rbac:groups=db.example.com,resources=postgresclusters/finalizers,verbs=update
4//+kubebuilder:rbac:groups=apps,resources=statefulsets,verbs=get;list;watch;create;update;patch;delete
5//+kubebuilder:rbac:groups=core,resources=services;configmaps;secrets;serviceaccounts,verbs=get;list;watch;create;update;patch;delete
6//+kubebuilder:rbac:groups=batch,resources=cronjobs,verbs=get;list;watch;create;update;patch;delete
7//+kubebuilder:rbac:groups=core,resources=events,verbs=create;patch

Run make manifests to regenerate RBAC YAML from these annotations. This keeps RBAC rules in sync with the code rather than requiring manual updates.

Operator Maturity Model

The Operator Framework defines capability levels:

Level	Capability	Example
1 — Basic Install	Automate installation and config	Deploy the StatefulSet and Services
2 — Seamless Upgrades	Manage version upgrades	Rolling upgrade of PostgreSQL minor version
3 — Full Lifecycle	Backup, restore, failure recovery	Automated backups, Patroni failover handling
4 — Deep Insights	Metrics, alerts, status	Custom metrics, degraded state detection
5 — Autopilot	Self-tuning, auto-scaling	Automatic resource request adjustment, replication factor optimisation

Don't try to reach level 5 on version 0.1. Start with Level 1 (install automation) and iterate based on operational pain points. Most production operators are level 3.

Using Existing Operators

Before building your own, check if a production-grade operator already exists:

PostgreSQL: CloudNativePG (CNCF Sandbox), Crunchy Data PGO
Redis: Redis Operator by OpsTree, Bitnami Redis with Sentinel
Kafka: Strimzi (CNCF)
Elasticsearch: Elastic ECK (official)
MongoDB: MongoDB Community Operator (official)
MySQL: Percona Operator for MySQL
Prometheus: kube-prometheus-stack (the CRDs themselves are managed by the operator)
Cert rotation: cert-manager (CNCF)
Secret sync: External Secrets Operator (CNCF Sandbox)

For standard infrastructure (databases, caches, message brokers), an existing mature operator is almost always better than building your own. The CloudNativePG team has years of production experience encoded in their operator that would take you months to replicate.

Build your own operator for domain-specific operational logic that existing operators don't cover — your proprietary application stack, platform primitives unique to your organisation, or cases where existing operators don't fit your requirements.

Frequently Asked Questions

Can I write an operator in a language other than Go?

Yes. The operator-sdk supports Go, Ansible, and Helm. There are also SDKs for Python (kopf), Java (java-operator-sdk), and Rust (kube-rs). Go is the canonical choice because it's what controller-runtime and client-go are written in — you get the best tooling, documentation, and community support. For operators with simple reconciliation logic, Helm-based operators (using Helm charts as the reconcile logic) are a good choice without Go expertise.

How do I handle CRD version upgrades?

CRD versions are immutable — you can't change field types in an existing version. Instead:

Add a new version (v1beta1) alongside the old (v1alpha1)
Implement conversion webhooks that convert between versions
Mark the old version as storage: false and new version as storage: true
Gradually migrate existing resources to the new version

kubebuilder's conversion webhook scaffold automates most of this.

How do I test a controller?

1// Use envtest — starts a local API server and etcd
2var _ = Describe("PostgresCluster Controller", func() {
3    It("Should create a StatefulSet", func() {
4        cluster := &dbv1alpha1.PostgresCluster{
5            ObjectMeta: metav1.ObjectMeta{
6                Name:      "test-cluster",
7                Namespace: "default",
8            },
9            Spec: dbv1alpha1.PostgresClusterSpec{
10                Version:  "16",
11                Replicas: 3,
12            },
13        }
14        Expect(k8sClient.Create(ctx, cluster)).To(Succeed())
15
16        // Wait for StatefulSet to be created
17        sts := &appsv1.StatefulSet{}
18        Eventually(func() error {
19            return k8sClient.Get(ctx, types.NamespacedName{
20                Name: "test-cluster", Namespace: "default",
21            }, sts)
22        }).Should(Succeed())
23
24        Expect(sts.Spec.Replicas).To(Equal(ptr.To(int32(3))))
25    })
26})

envtest runs a real API server in-process. Your controller runs against it like a real cluster. This gives you much higher confidence than mocking the Kubernetes API.

For Kubebuilder scaffolding and testing with envtest, see Kubernetes Operators: Building Controllers with Kubebuilder. For advanced controller-runtime patterns including Server-Side Apply, leader election, and production reconciler design, see Kubernetes Operators: Building Controllers with controller-runtime. For the GitOps patterns that manage operator installation and CRD updates, see GitOps with Argo CD: Production Setup Guide. For the IDP context where operators provide self-service infrastructure, see Platform Engineering: Building an Internal Developer Platform.

Building a Kubernetes operator for a complex stateful workload? Talk to us at Coding Protocols — we help platform teams design operator architectures that encode operational knowledge reliably.

Kubernetes Operators: Building Custom Controllers with CRDs