Build a Local AI Kubernetes Troubleshooting Agent with Ollama
Quick answer
Wire a tool-calling local model up to three read-only Kubernetes functions and watch it diagnose a broken pod on its own — no API key, no cluster data leaving your machine, and a token budget you control end to end.
- Step 1: Pull a Tool-Calling Model
- Step 2: Break Something on Purpose
- Step 3: Install Dependencies
- Step 4: Write the Tools
- Step 5: Describe the Tools for the Model
intermediate · 45 min
Before you begin
- Ollama installed and running — see Run an LLM Locally with Ollama if you haven't
- A tool-calling-capable model pulled (qwen3, gpt-oss, or llama3.1)
- Python 3.10+, and a cluster you can use (kind or minikube is fine) with kubectl configured
- A pod you can deliberately break, or a real broken one to point this at
The agent loop behind every "AI troubleshoots your cluster" demo is the same handful of lines: send the model a prompt and a list of tools, run whichever tools it asks for, feed the results back, repeat until it stops asking. Nothing about that loop requires a hosted model — it requires tool calling, which several models you can run on your own laptop now support.
This tutorial builds that loop against a local Ollama model instead of a cloud API. The payoff is direct: pod logs and event messages — which regularly contain things you'd rather not ship to a third party — never leave your machine, and there's no per-token bill while you iterate on the prompt and tools.
What You'll Build
- Three read-only Kubernetes tools:
get_pods,get_events,get_logs— the same shape as a cloud-backed agent, just called locally - A tool-calling loop against Ollama's OpenAI-compatible endpoint
- A broken pod to point it at, and a printed diagnosis you can sanity-check yourself
Step 1: Pull a Tool-Calling Model
Not every model Ollama serves supports tool calling — check the "tools" capability badge on a model's library page before you build around one. qwen3 and gpt-oss are Ollama's current featured tool-calling models; llama3.1 also still works if you already have it pulled:
ollama pull qwen3:8b8B is enough to follow a short, well-described tool list; if diagnoses come back vague or it stops calling tools too early, gpt-oss:20b is the heavier, more capable alternative before you look at switching model families entirely.
Step 2: Break Something on Purpose
You need a real failure to diagnose. This one OOMKills reliably:
1kubectl create namespace agent-demo
2kubectl apply -n agent-demo -f - <<'EOF'
3apiVersion: v1
4kind: Pod
5metadata:
6 name: oom-demo
7spec:
8 containers:
9 - name: stress
10 image: polinux/stress
11 command: ["stress"]
12 args: ["--vm", "1", "--vm-bytes", "150M", "--vm-hang", "0"]
13 resources:
14 limits:
15 memory: 50Mi
16EOFA manifest rather than kubectl run --limits=memory=50Mi — the --limits and --requests flags were dropped along with kubectl run's generators, so on any current kubectl that shortcut fails with error: unknown flag: --limits. The limit is the whole point here: 50Mi against a process reserving 150M is what makes the kernel kill it.
kubectl -n agent-demo get pods -w
# wait for restarts to climb — Ctrl-C once you see itStep 3: Install Dependencies
pip install openai kubernetesYou're using the openai package as a client library only — no OpenAI account, no API key that does anything. It talks to Ollama's local endpoint instead.
Step 4: Write the Tools
Same three read-only functions as any other Kubernetes agent — the interesting part in this tutorial is the loop, not the tools:
1# tools.py
2from datetime import datetime, timezone
3
4from kubernetes import client, config
5
6config.load_kube_config()
7core = client.CoreV1Api()
8
9MAX_LOG_BYTES = 20_000
10UNDATED = datetime.min.replace(tzinfo=timezone.utc)
11
12
13def get_pods(namespace: str, selector: str | None = None) -> str:
14 pods = core.list_namespaced_pod(namespace, label_selector=selector or "")
15 if not pods.items:
16 return f"No pods found in {namespace}"
17
18 lines = []
19 for p in pods.items:
20 statuses = p.status.container_statuses or []
21 restarts = sum(c.restart_count for c in statuses)
22 terminated = [
23 f"exit {c.last_state.terminated.exit_code} ({c.last_state.terminated.reason})"
24 for c in statuses
25 if c.last_state and c.last_state.terminated
26 ]
27 lines.append(
28 f"{p.metadata.name} phase={p.status.phase} restarts={restarts}"
29 + (f" {', '.join(terminated)}" if terminated else "")
30 )
31 return "\n".join(lines)
32
33
34def get_events(namespace: str, name: str | None = None) -> str:
35 field = f"involvedObject.name={name}" if name else ""
36 events = core.list_namespaced_event(namespace, field_selector=field)
37 rows = sorted(events.items, key=lambda e: e.last_timestamp or e.event_time or UNDATED)
38 return "\n".join(
39 f"{e.last_timestamp} {e.type} {e.reason}: {e.message}" for e in rows[-25:]
40 ) or "No events found."
41
42
43def get_logs(namespace: str, name: str, previous: bool = False) -> str:
44 try:
45 text = core.read_namespaced_pod_log(
46 name=name, namespace=namespace, previous=previous, tail_lines=200
47 )
48 except client.exceptions.ApiException as e:
49 return f"Could not read logs: {e.reason}"
50
51 if len(text) > MAX_LOG_BYTES:
52 text = "...[truncated]...\n" + text[-MAX_LOG_BYTES:]
53 return text or "(empty)"Step 5: Describe the Tools for the Model
Ollama's tool calling follows the OpenAI function-calling schema — a JSON Schema per function, separate from the function itself:
1# schema.py
2TOOLS = [
3 {
4 "type": "function",
5 "function": {
6 "name": "get_pods",
7 "description": "List pods with phase, restart count, and exit reason. Start here.",
8 "parameters": {
9 "type": "object",
10 "properties": {
11 "namespace": {"type": "string"},
12 "selector": {"type": "string", "description": "Optional label selector"},
13 },
14 "required": ["namespace"],
15 },
16 },
17 },
18 {
19 "type": "function",
20 "function": {
21 "name": "get_events",
22 "description": "Recent events for a namespace or object — explains OOM kills, scheduling failures, image pull errors.",
23 "parameters": {
24 "type": "object",
25 "properties": {
26 "namespace": {"type": "string"},
27 "name": {"type": "string", "description": "Optional object name to filter to"},
28 },
29 "required": ["namespace"],
30 },
31 },
32 },
33 {
34 "type": "function",
35 "function": {
36 "name": "get_logs",
37 "description": "Read container logs. previous=true reads the container that just crashed — that's where the error is for a restart loop.",
38 "parameters": {
39 "type": "object",
40 "properties": {
41 "namespace": {"type": "string"},
42 "name": {"type": "string"},
43 "previous": {"type": "boolean"},
44 },
45 "required": ["namespace", "name"],
46 },
47 },
48 },
49]Step 6: Write the Loop
This is the whole agent: send messages and tools, run whatever the model asks for, append results, repeat until it answers in plain text instead of a tool call.
1# agent.py
2import json
3import sys
4
5from openai import OpenAI
6
7from schema import TOOLS
8from tools import get_pods, get_events, get_logs
9
10client = OpenAI(base_url="http://localhost:11434/v1/", api_key="ollama")
11MODEL = "qwen3:8b"
12
13DISPATCH = {"get_pods": get_pods, "get_events": get_events, "get_logs": get_logs}
14
15SYSTEM_PROMPT = (
16 "You are a Kubernetes troubleshooting assistant. You have read-only tools: "
17 "get_pods, get_events, get_logs. Investigate using them before answering. "
18 "When you're confident, explain the root cause and a concrete fix in plain text."
19)
20
21
22def run(question: str) -> None:
23 messages = [
24 {"role": "system", "content": SYSTEM_PROMPT},
25 {"role": "user", "content": question},
26 ]
27
28 for _ in range(6): # hard cap — a local model that loops shouldn't run forever
29 response = client.chat.completions.create(
30 model=MODEL, messages=messages, tools=TOOLS,
31 )
32 msg = response.choices[0].message
33
34 if not msg.tool_calls:
35 print(msg.content)
36 return
37
38 messages.append(msg)
39 for call in msg.tool_calls:
40 args = json.loads(call.function.arguments)
41 print(f"[tool] {call.function.name}({args})", file=sys.stderr)
42 result = DISPATCH[call.function.name](**args)
43 messages.append({
44 "role": "tool",
45 "tool_call_id": call.id,
46 "content": result,
47 })
48
49 print("Gave up after 6 tool-call rounds without a final answer.")
50
51
52if __name__ == "__main__":
53 run(sys.argv[1])Step 7: Run It
python agent.py "oom-demo in agent-demo keeps restarting, why?"Watch stderr for the tool calls as they happen — that's the loop from Step 6 doing its job, not something hidden inside a framework:
[tool] get_pods({'namespace': 'agent-demo'})
[tool] get_events({'namespace': 'agent-demo', 'name': 'oom-demo'})
[tool] get_logs({'namespace': 'agent-demo', 'name': 'oom-demo', 'previous': True})
The container was OOMKilled: its memory limit is 50Mi, and the process is
deliberately allocating ~150Mi. Exit code 137 confirms the kernel killed it,
and the event log shows "OOMKilling" at the same timestamp. Raise the memory
limit above what the workload actually needs, or reduce the workload's
allocation.
If the model answers immediately without calling any tools, it's guessing from the pod name rather than investigating — tighten the system prompt to explicitly require calling get_pods first, and re-run.
Step 8: Clean Up
kubectl delete namespace agent-demoComparing This to a Cloud-Backed Agent
The loop is structurally identical to one built against a hosted model — same four-step shape, same tool functions, same idea of stopping when the model returns plain text instead of a tool call. What's different:
- No data leaves the machine. Pod logs and event messages, which can contain anything a workload happened to print, never cross a network boundary to a model provider.
- No per-token bill, which matters while you're iterating on tool descriptions and prompts — the phase where you'll run the loop dozens of times fixing wording.
- Smaller, less capable model. An 8B local model asks fewer follow-up questions and occasionally misreads ambiguous logs where a larger hosted model wouldn't. For genuinely hard incidents, that trade may not be worth it — know where the line falls before you commit to local-only for anything production-critical.
Where to Go Next
- Deploy this as a reusable service instead of a script — Build a Kubernetes MCP Server in Python moves these same three tools behind an MCP server with its own ServiceAccount, so any MCP client can use them, not just this one script.
- Scope the credential properly before pointing this at a real cluster — Scope RBAC for an AI Agent's Kubernetes Access builds the read-only identity this script's kubeconfig should actually be using.
- Move the model off your laptop once this is worth running unattended — Self-Host Ollama covers running it as a real service reachable from more than one machine.
- Read the cloud-model version of this same build — Build an AI Kubernetes Troubleshooting Agent covers the same loop against a hosted model, plus the RBAC and token-budget reasoning in more depth.
We built Podscape to simplify Kubernetes workflows like this — logs, events, and cluster state in one interface, without switching tools.
Struggling with this in production?
We help teams fix these exact issues. Our engineers have deployed these patterns across production environments at scale.