About the Prometheus Alert Rule Builder
A Prometheus alert rule is more than a PromQL expression: the for: duration decides flappiness, labels route it through Alertmanager, and annotations determine whether the on-call engineer at 3 a.m. understands the problem or just the symptom. This builder assembles complete, valid alerting rules with all four parts considered.
It encodes the patterns that separate quiet, trustworthy alerting from pager fatigue: alerting on symptoms rather than causes, rate windows sized to scrape intervals, and severity labels that map to real escalation behavior.
Frequently asked questions
What does the for: clause actually do?
The expression must remain true continuously for that duration before the alert fires — it is the flap damper. Too short and every transient spike pages; too long and real incidents go unnoticed. Typical values: 5–15m for symptom alerts, 0–1m only for severe, unambiguous conditions like "job absent". During the wait the alert is 'pending', visible in Prometheus but not yet routed.
Why do my rate() alerts never fire (or always fire)?
rate() needs at least two samples in its window — rate(x[1m]) with a 30s scrape interval is borderline and breaks when a scrape is missed. Rule of thumb: window ≥ 4× scrape interval (use [5m] for 30–60s scrapes). The opposite failure is comparing rate() output to thresholds meant for totals: rate() is per-second, so 0.5 means 30 errors/minute.
What labels and annotations should every alert carry?
Labels drive routing: severity (mapped to real paging behavior), team or service (so Alertmanager routes to the right receiver). Annotations carry human context: a summary with the affected instance via templating ({{ $labels.instance }}), and a runbook_url — an alert without a runbook link is a puzzle, not a page.
Need this managed for you, not just automated?
We're also a hands-on DevOps consultancy — Kubernetes, CI/CD, and cloud infrastructure.