Loki vs Elasticsearch

Loki and Elasticsearch answer different questions. Loki asks: what logs does this pod/namespace/label-set have, right now? It indexes only labels (pod name, namespace, container) and stores raw log lines compressed in object storage — making it dramatically cheaper than Elasticsearch at scale. Elasticsearch asks: across all my logs, show me every line containing 'OOMKilled' in the last hour, ranked by relevance. Its full-text inverted index enables queries Loki can't efficiently answer. The right choice depends on whether you're doing Kubernetes operations or log analytics.

Loki is the right choice for Kubernetes log aggregation when your primary use case is operational: reading logs for a specific pod, filtering by namespace, or detecting patterns in a known service. Storage costs are typically 80–90% lower than Elasticsearch for the same retention period because Loki compresses and ships chunks directly to S3 without indexing content. The PLG stack (Promtail + Loki + Grafana) integrates naturally with your existing Prometheus/Grafana dashboards. For most platform teams, Loki covers 90% of log query needs at a fraction of the cost.

Elasticsearch wins when you need to search arbitrary content across large log volumes, run structured analytics on log data, or build compliance reporting. Its full-text index answers queries like 'show all requests where response_time > 2s AND user_id = X' without scanning every log line. Kibana's Discover, Lens, and Canvas tools are significantly more powerful than Grafana's Explore for business-intelligence use cases. If your security team needs log correlation, your compliance team needs audit reports, or your application team runs complex analytics, Elasticsearch earns its operational cost.

Migrating from Elasticsearch to Loki requires replacing your log shippers (Filebeat → Promtail/Alloy), re-training your team on LogQL (simpler than KQL but different), and accepting that some complex queries you run today may be slower or impossible in Loki. Going the other direction — Loki to Elasticsearch — requires provisioning an Elasticsearch cluster (significant ops work), installing and configuring Filebeat, and replacing LogQL queries with KQL. Budget 1–2 weeks for a production migration in either direction. The LogQL learning curve deserves particular attention: engineers accustomed to Elasticsearch's query DSL need time to adjust to LogQL's label-filter syntax, and Grafana Explore's interface differs significantly enough from Kibana Discover that expect a productivity dip for the first few weeks.