Loki vs Elasticsearch for Kubernetes Logging: Cost, Queries, and the OpenShift Signal

Loki and Elasticsearch disagree about exactly one thing, and everything else follows from it: what deserves an index. Elasticsearch builds a full-text inverted index over every token of every log line, so any query is fast. Loki indexes only a small set of labels — namespace, pod, container — and stores the log content itself as compressed chunks in object storage, so ingest and storage are cheap and content search happens at query time.

That single decision explains the cost gap, the query trade-offs, the operational differences, and why Red Hat retired the EFK stack on OpenShift in favour of Loki. Pick based on which side of that decision matches how your team actually reads logs.

This post is the decision half of the story. The implementation half — Fluent Bit as a DaemonSet shipping into Loki with S3, LogQL patterns, multiline parsing — is covered in Kubernetes logging with Fluent Bit and Loki.

The architecture difference, concretely

Take one log line:

2026-08-12T09:14:03Z ERROR payment failed order=ord-8842 user=u-1932 gateway=stripe timeout after 30s

Elasticsearch tokenises it — error, payment, failed, ord-8842, u-1932, stripe, timeout — and adds every token to an inverted index alongside the document. Searching for ord-8842 across six months of logs is an index lookup: milliseconds, regardless of volume. The price is that the index frequently rivals or exceeds the size of the raw data, it must live on fast local disk (hot nodes on NVMe/EBS), and every line pays full indexing cost at ingest whether anyone ever searches it or not.

Loki stores that line, compressed, inside a chunk in S3. The only thing indexed is the label set of its stream — something like {namespace="payments", pod="payments-api-7d9f", container="app"}. The index is tiny (with the TSDB index format, typically well under 1% of log volume). Searching for ord-8842 means: narrow by labels and time range, pull the matching chunks from object storage, decompress, and grep in parallel across query workers. LogQL calls this a line filter:

{namespace="payments"} |= "ord-8842"

The mental model that sticks: Elasticsearch is a search engine that happens to store logs. Loki is a log store that happens to support search. Loki's own tagline — "like Prometheus, but for logs" — is accurate: streams identified by label sets, content scanned on demand.

What that means for cost

The cost difference is not a percentage — it is an order of magnitude, and it comes from two compounding factors.

Storage class. Loki's chunks live in S3 at ~$0.023/GB-month, with lifecycle transitions to cheaper tiers for long retention. Elasticsearch's hot tier lives on block storage attached to running nodes — gp3 EBS at ~$0.08/GB-month — plus the EC2 instances that must stay up to serve it. ES has searchable snapshots and frozen tiers backed by object storage now, which narrows the gap for cold data, but the hot/warm window where most querying happens still rides on provisioned nodes.

Write amplification. Elasticsearch stores the source document and the inverted index; with replicas and analysed fields, 1TB of raw logs commonly becomes 1.2–2TB on disk. Loki compresses chunks (Snappy/gzip) at typically 5–10x, so the same 1TB becomes 100–200GB in S3, and the index adds a rounding error.

A worked example: 200GB/day, 30-day retention

A mid-size EKS platform emitting 200GB of raw logs per day, 30-day hot retention. Numbers are illustrative us-east-1 list prices — run your own, but the shape holds.

Roughly 4–5x here, and the gap widens with retention: extending Loki to 90 days adds ~$40/month of S3; extending Elasticsearch's searchable window means more disk and usually more nodes. Managed Elasticsearch (Elastic Cloud, OpenSearch Service) shifts the line items but not the ratio. This is the same "storage class is destiny" dynamic that shows up in observability cost for AI workloads — verbose logging multiplied by per-GB indexed pricing is how observability bills quietly pass compute bills.

Query trade-offs: where each one wins

Cheap storage is not free — you pay at query time, and honesty about that is what separates a good decision from a migration you regret.

Where Elasticsearch wins: needle-in-haystack. "Find this request ID somewhere in the last 90 days, I don't know which service" is Elasticsearch's home turf — one index lookup, instant, over any time range. In Loki that same query with a wide label matcher and a long range means downloading and scanning a large fraction of your chunks. Query-parallelisation and caching make it tolerable (seconds to minutes, not hours), but it will never beat an inverted index at its own game. Fuzzy matching, relevance scoring, aggregations over arbitrary text fields — all Elasticsearch, no contest.

Where Loki wins: scoped incident debugging. Most real production queries are not needle-in-haystack. They are "show me errors from the payments namespace in the last hour":

{namespace="payments", container="payments-api"} |= "ERROR"

The label match narrows this to a handful of streams and a one-hour window before any scanning happens — fast in practice, and next to metrics and traces in the same Grafana pane. LogQL's metric queries (rate({...} |= "ERROR" [5m])) turn log content into alertable time series via the ruler, which covers a lot of what people build painful Watcher/ElastAlert setups for.

Loki's foot-gun: label cardinality. Every unique label combination is a separate stream with its own chunks and index entries. Put user_id, request_id, or trace_id into labels and you create millions of tiny streams — index bloat, ingester memory pressure, and eventually a fallen-over cluster. The rule is strict: labels describe where a log came from, not what it says. High-cardinality values stay in the log line (or structured metadata) and get filtered at query time. Teams coming from Elasticsearch, where indexing everything is the whole point, hit this wall hardest.

Operational burden

Elasticsearch is a distributed stateful database, and it makes you run it like one. Shard sizing (the eternal 10–50GB-per-shard guidance), ILM policies rolling indices through hot → warm → cold → delete, JVM heap tuning, mapping explosions from a team that logged a new nested JSON shape, the red-cluster-at-2am unassigned-shards drill. None of it is exotic, but all of it is somebody's recurring job. Managed offerings absorb the hardware layer; shard strategy, mappings, and ILM remain yours.

Loki moves state to object storage, which removes the scariest failure modes. Writers keep a short WAL; nearly everything else sits statelessly in front of S3 — losing a read pod loses nothing. Deployment modes scale with you: monolithic (one binary, fine to ~20GB/day), simple scalable (separate read/write/backend targets — the long-running production default, though Grafana has now deprecated it with removal planned for Loki 4.0, steering new installs toward HA monolithic or microservices), and microservices (fully decomposed, for very large multi-tenant installs). Loki has its own tuning surface — per-tenant ingestion limits, compactor retention, query parallelism, and the cardinality discipline above — but "misconfigured limits return a 429" is a categorically better failure mode than "cluster is red and writes are rejected."

Ecosystem fit

Loki is Grafana-native. Logs land in the same Explore view as Prometheus metrics and Tempo traces, derivedFields turns trace IDs in log lines into click-through links, and alerting flows through the same Alertmanager pipeline you already run for metrics. If your stack is the one described in Prometheus and Grafana on Kubernetes — or you're feeding everything through the OpenTelemetry Collector, which ships logs to either backend — Loki completes a coherent single-vendor-free story: metrics, logs, and traces correlated in one UI, alerting in one place.

Elasticsearch brings Kibana — genuinely excellent log search UX, plus the wider Elastic platform: SIEM detection rules, anomaly detection ML jobs, APM. If your security team lives in Elastic SIEM, that alone can decide the question, because Loki is not a SIEM and does not pretend to be. The awkward middle is running Kibana for logs and Grafana for metrics: two UIs, two alerting systems, two RBAC models, and incident responders tab-switching between them.

The OpenShift signal

If you want a large-scale, production-weighted opinion on this question, Red Hat already published one. OpenShift's cluster logging was EFK — Elasticsearch, Fluentd, Kibana — for years. Red Hat deprecated that stack and replaced it with LokiStack (the Loki Operator) fronted by the Vector-based collector, with logs viewed in the OpenShift console; OpenShift Logging 6.x dropped the Elasticsearch/Fluentd/Kibana operators entirely.

The stated reasoning matches everything above: the bundled Elasticsearch was the heaviest, most failure-prone component of cluster logging — memory-hungry, storage-bound to PVs — while Loki against object storage is dramatically lighter to embed and support. Read it precisely: Red Hat did not declare Elasticsearch bad at search. They declared it the wrong default for platform logging, where the workload is "collect everything, query recent slices during incidents" rather than "search everything, always." If an OpenShift upgrade is forcing the move on you: the direction is one-way, EFK is not coming back, and the migration path below applies with the operator handling the Loki half.

When Elasticsearch is still the right answer

The honest list, because it exists:

• Search-heavy compliance and audit. Auditors asking "every action by user X across 13 months, now" is exactly the arbitrary-field, long-range, interactive search an inverted index is for.
• SIEM and security analytics. Correlation rules, threat-intel enrichment, ML anomaly jobs over full-text — the Elastic Security stack has no Loki equivalent.
• Log search as a product feature. If users search logs in your product, relevance scoring and sub-second arbitrary queries are requirements, not luxuries.
• A real existing ELK investment. Years of Kibana dashboards, ES-integrated tooling, and a team fluent in its operation — migration cost is real, and "cheaper storage" alone may not repay it on your timeline.
• Genuinely unpredictable query patterns across long ranges, all day, by many teams. Loki's scan-at-query-time economics assume most queries are scoped; if yours aren't, you're just moving cost from storage to query compute and patience.

Migration sketch: ES to Loki without a cliff

The pragmatic path is parallel running, not a cutover:

1. Dual-ship. Your collector fans out — Fluent Bit, Fluentd, and the OTel Collector all support multiple outputs. Add a Loki output next to the Elasticsearch one; the Fluent Bit → Loki setup is a values-file change, with Loki in Simple Scalable mode (still supported throughout 3.x, despite the 4.0 deprecation) against a fresh S3 bucket.
2. Design labels deliberately. Around 10–15 low-cardinality labels: namespace, app, container, cluster, environment. Nothing request-scoped. This step decides whether Loki works for you.
3. Rebuild the queries people actually run. Translate saved Kibana searches to LogQL; recreate log-driven alerts as Loki ruler rules. Expect a couple of weeks of "how do I write X in LogQL" friction.
4. Shrink ES retention instead of killing ES. Drop hot retention to 7 days while Loki holds 30–90. Costs fall immediately; the escape hatch stays open.
5. Decide the endgame. Many teams land on Loki for platform logs plus a small ES cluster for the audit/SIEM slice — a deliberate, cheap split rather than an accidental two-stack sprawl.

Historical logs generally aren't worth back-migrating; let ES age out its window and treat the Loki start date as day zero.

Head to head

Frequently Asked Questions

Is Loki really 10x cheaper than Elasticsearch?

For storage, routinely — compressed chunks in S3 versus index-amplified data on EBS-backed nodes is an order-of-magnitude gap on the per-GB line, and total cost lands around 3–5x lower once compute is included (see the worked example above). The gap grows with retention. What you give up is instant arbitrary full-text search over long ranges.

Why did OpenShift move from EFK/Elasticsearch to Loki?

Red Hat deprecated the EFK stack and made LokiStack the supported logging store because the bundled Elasticsearch was the heaviest, most operationally fragile part of cluster logging, while Loki against object storage is far cheaper to run and support as a platform default. OpenShift Logging 6.x removed the Elasticsearch, Fluentd, and Kibana components entirely.

Can Loki do full-text search?

At query time, yes — |= "text" and |~ "regex" scan chunk content, parallelised across queriers. What it cannot do is indexed full-text lookup: no relevance ranking, no fuzzy matching, and wide-range searches cost time proportional to data scanned. Scoped searches (namespace + hours) feel instant; "all logs, 90 days" does not.

What is the biggest mistake teams make adopting Loki?

High-cardinality labels. Promoting user_id, request_id, or trace_id to labels creates a stream per unique value and melts the ingesters. Keep labels to stable topology (namespace, app, container, environment) and filter everything else at query time.

Should I run both?

It's a legitimate endgame, not a failure: Loki for high-volume platform and application logs, a deliberately small Elasticsearch for audit search or SIEM. That mirrors the hybrid pattern in Prometheus vs Datadog — cheap OSS for the bulk, the specialised tool where its strengths are actually load-bearing.

What about OpenSearch?

Everything here applies to OpenSearch equally — inverted index, hot nodes, shards, ILM (ISM in OpenSearch). The fork changed licensing, not the storage economics.

The decision in one paragraph

Count your queries. If the overwhelming majority are "recent logs from a known service, during an incident" — which is most Kubernetes platform teams — Loki gives you that experience inside Grafana at a fraction of the cost, and OpenShift's move tells you the platform vendors have done the same math. If your organisation genuinely searches — compliance trawls, security hunting, arbitrary text across months — Elasticsearch's index is worth every gigabyte it costs. And if you have both workloads, split them deliberately instead of forcing one tool to be bad at half its job.

See also

• Kubernetes Logging: Fluent Bit and Grafana Loki — the full implementation of the Loki side of this comparison
• Prometheus and Grafana on Kubernetes — the metrics stack Loki slots alongside
• OpenTelemetry on Kubernetes — one collector pipeline feeding either backend
• Monitoring Strategy: Prometheus vs Datadog — the same OSS-vs-managed calculus, for metrics
• Observability Cost for AI Workloads — what happens when log volume meets per-GB pricing

Official References

• Loki documentation — label design, LogQL and retention
• Grafana documentation — dashboards, panels, variables and provisioning