Elasticsearch Index Management: 6 Problems Hiding in Your Cluster

Your Elasticsearch cluster went yellow six months ago. Someone checked, said "it's just a replica," and closed the tab. The dashboard tile has been amber ever since, and at this point it would be more alarming if it turned green.

That normalized yellow is the signature failure mode of Elasticsearch index management: the cluster keeps answering queries, so nobody treats degradation as a problem until search latency doubles or a node hits the flood-stage watermark and indexing stops mid-afternoon. Between "working" and "down" there is a long, expensive middle where the cluster is slower and larger than it needs to be — and every symptom in that middle traces back to how indices are created, sized, and retired.

This guide covers the six problems we see most on self-managed and Elastic Cloud clusters alike: what each one is, why it happens, one API call to detect it, and what it typically costs in latency and infrastructure. This is part one of a three-part series. Part two is a hands-on tour of index management with native Elasticsearch tools — the _cat APIs, ILM policies, slow logs. Part three covers automating the whole loop with an AI agent.

1. Cluster health stuck in yellow

What it is. Yellow means every primary shard is assigned but at least one replica is not. Queries still work, which is exactly why teams stop caring. But a yellow cluster has no redundancy for the affected indices: lose the node holding those primaries and you lose data, not just capacity. Yellow is also frequently a symptom of something else on this list — a disk watermark blocking allocation, or more shards than the nodes can hold.

Why it happens. Replica counts set higher than the node count can satisfy (2 replicas on a 2-node cluster will never go green), a node that left and never came back, or allocation rules that quietly exclude the only eligible node. Nobody investigates because "it still works."

How to detect it.

curl -s "localhost:9200/_cluster/health?pretty"

Look past status. unassigned_shards tells you the size of the problem; initializing_shards stuck above zero for hours means allocation is wedged, not in progress. When you're ready to find out why a shard is unassigned, the cluster allocation explain API gives you the exact reason — we walk through it in part two.

Typical impact. Zero — until a node fails, at which point it's data loss on every index that had its only copy there. Treat sustained yellow as an incident with a slow fuse.

2. Oversharding and tiny-shard sprawl

What it is. Thousands of shards holding a few megabytes each. Every shard is a full Lucene index with its own segment files, memory overhead, and entry in the cluster state. A search across an oversharded index fans out to every shard and pays coordination cost on each; the master node re-publishes a bloated cluster state on every change. Elastic's own guidance is to aim for shards between 10 GB and 50 GB — most sprawled clusters average under 1 GB.

Why it happens. Daily indices created per-service, per-environment, per-customer, each with the old default of 5 primaries and 1 replica. Ten services × 365 days × 10 shards is 36,500 shards a year, most nearly empty.

How to detect it.

curl -s "localhost:9200/_cat/shards?v&h=index,shard,prirep,state,store,node&s=store:asc" | head -50

Sorting by store size ascending puts the junk on top. If the first screen is full of shards under 100 MB, you're oversharded. Cross-check the total against your heap: a common rule of thumb is to stay well under roughly 20 shards per GB of heap on each node.

Typical impact. This is the big one for both latency and cost. Consolidating tiny daily indices into rollover-managed ones commonly cuts shard counts by 80–90%, drops search p99 noticeably on multi-index queries, and frees enough heap to remove a node or two — often 15–30% of cluster infrastructure cost.

3. Indices that never roll over

What it is. An index lifecycle management (ILM) policy that exists but was never attached, or was attached with a rollover action that can never trigger — usually because the index wasn't created through an alias or data stream, so there is nothing to roll over. The index grows forever: 800 GB, one shard, no delete phase ever firing. The cluster stores every log line since 2023 on hot-tier NVMe.

Why it happens. ILM has real setup requirements — an index template, a write alias or data stream, and the policy — and if any leg is missing, ILM fails silently or reports an error nobody reads. The index keeps accepting writes, so the failure is invisible until the disk isn't.

How to detect it.

curl -s "localhost:9200/logs-*/_ilm/explain?only_errors=true&pretty"

An empty result here is not a clean bill of health — also run it without only_errors and scan for "managed": false (no policy attached at all) and for indices sitting in the same action for weeks. The ILM documentation covers what each phase and step should look like when healthy.

Typical impact. Retention is your single biggest storage lever. Clusters that fix broken ILM typically discover they are storing 2–5× the data their stated retention policy implies — pure disk and, on hot/warm architectures, pure hot-tier spend.

4. Slow logs nobody enabled

What it is. Elasticsearch ships with search and indexing slow logs — per-index logs that capture any query or indexing operation exceeding thresholds you set. By default, no thresholds are set, so nothing is logged. When search "feels slow," teams have exactly zero evidence about which queries, which indices, or which patterns are responsible, and the investigation starts from guesswork.

Why it happens. Slow logs are opt-in and per-index, so they never get enabled proactively — only mid-incident, when it's too late to capture the offending query.

How to detect it. Check whether any index has thresholds configured:

curl -s "localhost:9200/_all/_settings/index.search.slowlog*?pretty"

If the response comes back empty for every index, you're flying blind. Enabling them is one settings call with thresholds like index.search.slowlog.threshold.query.warn: 2s — exact syntax and how to read the output are in the slow log docs and in part two of this series.

Typical impact. Indirect but large: the difference between a two-hour incident ("we can see the exact wildcard query from the reporting service") and a two-day one. Elasticsearch slow queries are almost always a handful of repeat offenders — leading-wildcard searches, huge aggregations, deep pagination — and slow logs name them immediately.

5. Disk watermark pressure and flood-stage lockout

What it is. Elasticsearch protects itself with three disk thresholds (disk-based shard allocation): at the low watermark (85% by default) it stops allocating new shards to the node; at the high watermark (90%) it actively relocates shards away; at flood stage (95%) it slaps index.blocks.read_only_allow_delete on every index with a shard on that node, and writes start failing cluster-wide for those indices. Ingest pipelines back up, apps throw 429s, and the on-call gets paged for what is fundamentally a retention problem.

Why it happens. Usually problem #3 — indices that never roll over or delete — meets steadily growing log volume. Watermark breaches are the symptom that finally makes broken ILM visible.

How to detect it.

curl -s "localhost:9200/_cat/allocation?v&h=node,disk.percent,disk.used,disk.avail,shards"

Any node above 85% is already distorting shard placement; above 90% it's shuffling data around under load. Check this weekly, not after the page.

Typical impact. Flood stage is a production outage for ingest. Even below that, high-watermark relocations consume disk and network I/O that directly competes with search — clusters running hot on disk routinely see elevated latency for hours during rebalancing.

6. Mapping explosions from dynamic fields

What it is. With dynamic mapping on (the default), every new JSON key in an ingested document becomes a new field in the index mapping. Log payloads that use user IDs, request IDs, or arbitrary labels as keys — a fresh user_12345 object key in every document — generate a new field per document. Mappings balloon to tens of thousands of fields, the cluster state grows with them, and eventually indexing fails against the index.mapping.total_fields.limit (default 1,000).

Why it happens. Some upstream service serializes a map with dynamic keys, and nobody notices until either indexing breaks or the master starts struggling to publish cluster state. Raising the field limit "to fix it" makes the explosion legal, not gone.

How to detect it. Count fields on a suspect index:

curl -s "localhost:9200/logs-app-000001/_field_caps?fields=*" | jq '.fields | length'

Anything drifting toward four digits on a logging index deserves a look at the mapping itself (GET logs-app-000001/_mapping) to find the exploding branch. The fix is dynamic: false or flattened fields on the offending object — see mapping limit settings.

Typical impact. Degraded cluster-state publishing, heap pressure on every node, and hard indexing failures when the limit hits. On busy logging clusters, a single misbehaving service can take ingest down for everyone sharing the index template.

The pattern: these problems create each other

Notice the chain. ILM that never attaches (#3) grows indices until disks hit watermarks (#5), which blocks allocation and strands replicas, which parks the cluster in yellow (#1), which everyone ignores because search still works — just slower, because the index has one enormous shard while the rest of the cluster drowns in tiny ones (#2), and there are no slow logs (#4) to prove any of it.

That's why point-in-time checks age so badly here. Each of the six commands above takes seconds to run, but the answers change weekly: every new service adds index patterns, every traffic bump accelerates rollover math, every schema change can start a mapping explosion. Cluster hygiene isn't a project you finish; it's a rate you have to keep up with.

Start by running all six checks today — most teams find at least three findings on the first pass. Then read part two, where we do the full DIY version with native tools: _cat API workflows, writing and attaching an ILM policy that actually rolls over, enabling slow logs, and debugging unassigned shards with allocation explain.

Make the checks continuous

Everything above can run unattended. CloudThinker connects to Elasticsearch with a read-only API key and continuously watches cluster health, shard balance, ILM progress, and slow logs — correlating findings (the yellow status, the watermark, the index that never rolled over) instead of leaving you six separate graphs, with any fix gated behind your approval. Try CloudThinker free — 100 premium credits, no card required — or continue with the hands-on native-tools guide.