Give an AI Agent Read-Only Access to Kubernetes

Give an AI Agent Read-Only Access to Kubernetes

"Just give it read-only access" is the standard answer when someone proposes pointing an LLM at a production cluster. It sounds obviously safe, and the RBAC part of it genuinely is — Kubernetes has had a good answer for scoping a credential for a decade.

The problem is that read-only answers a question about writes, and an agent changes what you should be worried about.

Two things are different when the client holding the credential is a language model:

1. The client can be steered by the data it reads. Pod logs, event messages, and ConfigMap values are attacker-controllable in any cluster running user workloads. They arrive in the agent's context window as text, indistinguishable in kind from your instructions.
2. Every byte it reads leaves the cluster. A human with view looks at something and closes the terminal. An agent forwards it to a model provider, where it lands in request logs and retention windows — and usually into your own traces on the way.

So the operative question isn't "can it write?" It's "what can it see, and where does that end up?"

This post covers what changes for an agent. It assumes you already know how RBAC works — if not, Kubernetes RBAC in Practice is the foundation, Kubernetes RBAC verbs is the reference for which verbs grant what, and advanced RBAC patterns covers aggregation, escalation paths, and projected tokens in depth. The agent this is protecting is in Build an AI Kubernetes Troubleshooting Agent.

Reads are the risk surface now

Standard threat modelling ranks Kubernetes permissions roughly by destructiveness: delete is worse than create, create is worse than get. For an agent, re-rank by sensitivity of what's returned, because a successful get is the exfiltration event.

That inverts two habits.

secrets is not a read. Granting get on Secrets to an agent means copying every credential in scope into a context window that is transmitted to a third party, retained under their policy, possibly echoed back in the response text, and very likely captured in your tracing as a tool result. The built-in view role excludes Secrets for exactly this reason — that decision was made for humans and it's even more correct here.

configmaps is barely better. In a large enough cluster, ConfigMaps contain connection strings, internal hostnames, API endpoints, and a depressing number of credentials that should have been Secrets. Leave them out of the default grant. If the agent needs one specific ConfigMap, name it:

  - apiGroups: [""]
    resources: ["configmaps"]
    resourceNames: ["app-tuning-params"]
    verbs: ["get"]

resourceNames can't restrict a top-level create or a deletecollection at all, and restricting list by name only takes effect if the client sends a matching metadata.name field selector. Granting nothing but get sidesteps both, which is the right shape anyway — the agent has to know what it's asking for rather than enumerating the namespace.

The resulting role is deliberately boring:

1apiVersion: rbac.authorization.k8s.io/v1
2kind: ClusterRole
3metadata:
4  name: agent-readonly
5rules:
6  - apiGroups: [""]
7    resources: ["pods", "events", "services", "nodes", "namespaces"]
8    verbs: ["get", "list"]
9  - apiGroups: [""]
10    resources: ["pods/log"]
11    verbs: ["get"]
12  - apiGroups: ["apps"]
13    resources: ["deployments", "replicasets", "statefulsets", "daemonsets"]
14    verbs: ["get", "list"]
15  - apiGroups: ["batch"]
16    resources: ["jobs", "cronjobs"]
17    verbs: ["get", "list"]

Bind it with a RoleBinding per namespace rather than a ClusterRoleBinding, unless the agent genuinely needs cluster-wide reach. In a multi-tenant cluster, namespace scoping is what stops one team's agent from forwarding another team's logs to a model provider.

Note the trade in that first rule: nodes and namespaces are cluster-scoped, so a RoleBinding grants nothing for them however the ClusterRole is written. Either drop them and keep the binding namespaced, or split them into a second ClusterRole with a ClusterRoleBinding and be deliberate about it — don't leave them in a namespaced binding and assume they work.

Don't bind an agent to view

view looks like the obvious choice and it's the one I'd avoid.

view is an aggregated role: its rules are assembled at runtime from every ClusterRole labelled rbac.authorization.k8s.io/aggregate-to-view: "true". Any operator you install later can contribute rules to it — cluster-wide, for every subject bound to it, with no change to your manifests and nothing in your Git history.

For a human on-call engineer that's a minor concern. For an agent whose entire safety argument is "it only has view," it means the ceiling you verified at deploy time is not the ceiling six months later. Install a CRD whose status subresource carries something sensitive, labelled for view aggregation, and your agent can now read it.

# What's currently feeding the aggregate
kubectl get clusterroles \
  -l rbac.authorization.k8s.io/aggregate-to-view=true -o name

An explicit role you own is more verbose and doesn't move without you. That trade is worth making here in a way it usually isn't. (The aggregation mechanics themselves are covered in advanced RBAC patterns — the agent-specific point is just that drift you'd tolerate for a person is drift you shouldn't tolerate for an automated reader.)

Assume the token leaks

Agent credentials end up in places normal credentials don't: a prompt, a log line, a stack trace in an error message sent to a model provider, a debugging session someone pasted into a chat.

Plan for it. Use short-lived, audience-bound tokens rather than anything static:

kubectl create token k8s-agent \
  --namespace agents --duration=15m

Audience is the part people get wrong. The API server only accepts a service account token whose audience is one of its own --api-audiences, which is what kubectl create token requests when you omit the flag — pass --audience=kubernetes-agent and you get a token the cluster will refuse. Bind a custom audience when the agent presents the token to something other than the API server (a proxy, a sidecar, a cloud identity provider); there, an audience-bound token is useful in fewer places if it leaks. In-cluster, use a projected token volume and let the kubelet rotate it. On cloud clusters, prefer the platform identity mechanism over any static credential — IRSA and Pod Identity on AWS, workload identity elsewhere.

Verify the ceiling in CI

The system prompt saying "you have read-only access" is a request, not a control. The control is that the credential physically cannot do more, and that's assertable:

1SA=system:serviceaccount:agents:k8s-agent
2
3for check in "delete pods" "create pods --subresource=exec" "get secrets" \
4             "create serviceaccounts --subresource=token" "patch deployments"; do
5  if kubectl auth can-i $check --as=$SA --quiet 2>/dev/null; then
6    echo "FAIL: agent can $check"; exit 1
7  fi
8done
9echo "OK: agent ceiling holds"

--quiet sets the exit code instead of printing, which makes it scriptable. Subresources go through --subresource, not a slash: can-i create pods/exec is parsed as TYPE/NAME — a pod named exec — so it passes while telling you nothing about exec. Run it after every RBAC change and after every operator install — the aggregation problem above means an unrelated Helm chart can move your ceiling without touching your RBAC manifests.

Those five checks are the security boundary. Everything in the prompt is a preference.

Invert your audit policy

This is the change most teams miss.

Standard audit policy logs writes at RequestResponse and reads at Metadata or not at all — sensible, when reads are cheap and boring. For an agent, the read is the event you care about, because it's the moment data left the cluster.

1apiVersion: audit.k8s.io/v1
2kind: Policy
3rules:
4  # Expected agent behaviour — log it fully, this is your egress record.
5  - level: RequestResponse
6    users: ["system:serviceaccount:agents:k8s-agent"]
7    verbs: ["get", "list"]
8    resources:
9      - group: ""
10        resources: ["pods", "pods/log", "events"]
11
12  # Anything else from this identity is, by definition, unexpected.
13  - level: RequestResponse
14    users: ["system:serviceaccount:agents:k8s-agent"]

The second rule is the one that earns its place. The agent should only ever perform the reads in rule one. Anything else it manages to attempt — including the 403s from a prompt-injection attempt — is a finding worth alerting on, not a line to file.

This also answers the question that actually gets asked after an incident: what did the agent see? Without a read-level audit trail you are guessing, and "we're fairly sure it only read pod logs" satisfies nobody.

What this buys, and what it doesn't

It buys a hard ceiling. A malicious log line can convince the agent to try to delete a Deployment. The API server returns 403, the attempt lands in your audit log, and nothing happens. That is the correct outcome and no amount of prompt engineering produces it.

It does not stop data egress you authorised. Scoping controls what's reachable; it does nothing about what happens after. If pod logs in a namespace contain customer PII, a perfectly-scoped read-only agent is a PII egress path with an excellent ceiling. The fix there is upstream — stop logging the PII — not in the RoleBinding.

It does not make the agent's output safe. A confident, wrong diagnosis that sends an engineer to restart the wrong service causes an outage the credential had nothing to do with. The permissions are a containment story, not a correctness story.

When the request comes to add a write verb — and it will, as "can it just restart the pod?" — treat it as a design change rather than a permissions tweak. The answer is an approval-gated path with a human in it, not a wider ClusterRole. For what happens when roles grow the other way instead, RBAC misconfigurations that break production is the catalogue of where it ends up.