Automating Elasticsearch Index Management with AI Agents

There is a real difference between a Slack alert that says cluster status: yellow and a message that says: the cluster went yellow at 03:12 because 42 replica shards can't allocate — node es-data-4 crossed the low disk watermark, and the disk filled because logs-app-000001 has grown to 1.4 TB without ever rolling over. Its ILM policy is attached but stuck on check-rollover-ready because the write alias points at the wrong index. Here's the fix; approve to apply.

The first message is a fact. The second is an investigation. In Elasticsearch index management, most teams have plenty of the first kind and produce the second kind manually, at whatever hour the page arrives.

This is part three of our Elasticsearch index management series. Part one mapped the signals that matter — yellow-forever clusters, oversharding, stuck ILM, slow logs nobody enabled, disk watermarks, mapping explosions. Part two covered the DIY toolkit: _cat APIs, ILM policies, slow logs, allocation explain, snapshot lifecycle management. If you followed along, you can now diagnose everything in that opening paragraph yourself. The catch is that you have to keep doing it — every incident, every quarter, forever.

This article covers the alternative: CloudThinker agents as a continuous action layer on top of your cluster, doing the watching, the correlating, and — with your approval — the fixing.

Connecting: a read-only API key, about five minutes

CloudThinker connects to Elasticsearch the way your security review would demand anyway: an API key scoped to monitoring privileges, no write access, no plugin installed on the cluster. You create the key yourself, so you can see exactly what is granted:

POST /_security/api_key
{
  "name": "cloudthinker-readonly",
  "role_descriptors": {
    "cloudthinker_monitor": {
      "cluster": ["monitor", "read_ilm", "read_slm"],
      "indices": [
        {
          "names": ["*"],
          "privileges": ["monitor", "view_index_metadata"]
        }
      ]
    }
  }
}

Paste the endpoint URL and the encoded key into the connection screen, and the first scan starts. The whole flow takes about five minutes — the Elasticsearch connection guide walks through it for self-managed clusters, Elastic Cloud, and OpenSearch-compatible endpoints.

What the agents watch continuously

Everything from parts one and two, on a loop instead of a calendar:

• Cluster health and allocation — not just green/yellow/red, but why: unassigned shard counts, allocation-explain reasons, watermark proximity per node.
• Shard geometry — shard counts versus data volume, median shard size, tiny-shard sprawl, hot nodes carrying disproportionate shares.
• ILM progress — every managed index's phase, action, and step, flagging anything parked in the same step long enough to indicate a stuck policy rather than a slow one.
• Slow logs — search and indexing slow logs, parsed and grouped by query shape, so Elasticsearch slow queries surface as patterns ("wildcard queries against message.keyword on the biggest index") instead of ten thousand raw log lines.
• Snapshot health — SLM policy success and the age of the last good snapshot, because the day you need one is the wrong day to learn it's been failing for three weeks.

The cadence is the point. A cluster that went yellow on Tuesday is a correlated finding on Tuesday — not a normalized dashboard color that someone finally questions during the next incident.

An investigation, end to end

Here's what "investigate" means concretely, using the opening example. A dashboard shows you each of these facts on a different screen; the agent walks the chain:

1. _cluster/health reports yellow with 42 unassigned shards.
2. _cluster/allocation/explain on one of them says allocation is blocked because the target node exceeded the low disk watermark.
3. _cat/nodes confirms es-data-4 at 87% disk while its peers sit near 60% — so this is one index's weight, not general growth.
4. _cat/indices sorted by store size finds logs-app-000001 at 1.4 TB — two orders of magnitude past its intended rollover size.
5. _ilm/explain on that index shows a policy attached but stuck at check-rollover-ready: the rollover condition can never trigger because the write alias was pointed at a different index during a past migration.

The finding you receive is that chain, with evidence at each link, plus a proposed remediation: repoint the write alias, trigger a manual rollover, then reindex the oversized index into properly sized backing indices during a low-traffic window. Same skills you built in part two — minus the two hours of running them under pressure.

Graduated autonomy: nothing changes without your say-so

The connection is read-only by default, so at first the agents can only observe and explain. When you want them to act, you grant scoped write access and set an autonomy level per action type and environment:

• Notify — report the finding and the evidence. The default for everything.
• Suggest — propose the specific change: the ILM policy JSON, the reindex plan, the corrected shard count for the next rollover, with expected impact and rollback notes.
• Approve — the agent stages the action and executes only after a named human approves it.
• Autonomous — execute and report. Teams typically reserve this for reversible hygiene on non-production clusters — deleting empty indices past retention, say — after weeks of watching the agent be right at the Approve level.

Every finding, proposal, approval, and execution lands in an audit trail: what was observed, what was proposed, who approved it, what changed, when. Escalation paths stay intact — the agent files the investigation; it doesn't silence the pager.

What a first scan typically finds

Illustrative numbers from a composite mid-market logging cluster — six data nodes, roughly 8 TB of data, a few years of organic growth. Yours will differ:

Notice the shape: nothing exotic. It's the same hygiene list from part one — the value is that each row arrives already correlated, with a proposed fix attached, and the table refreshes itself instead of waiting for the next audit.

Prompts to try in your first session

The agents are conversational — you ask in plain language and get answers grounded in your cluster's actual API responses:

"Why has the cluster been yellow since Tuesday, and which index is responsible?"

"Show me the slowest search queries from the past week, grouped by query pattern, and which indices they hit."

"Draft an ILM policy for the logs-app indices — rollover at 50 GB or 7 days, move to warm after 3 days, delete after 30 — and show me which existing indices it would affect."

Each answer cites the underlying evidence — the same _cat, _ilm/explain, and slow-log data from part two — so you verify rather than trust.

What the agents will not do

Worth stating plainly, because "AI with access to the cluster that holds your logs" deserves scrutiny:

• They will not write anything under the default connection. The API key you created grants monitoring privileges only; remediation requires you to explicitly grant scoped write access and raise the autonomy level above Notify.
• They will not delete indices, change an ILM policy, or start a reindex without approval unless you deliberately set that action class to Autonomous for that environment.
• They will not act outside the audit trail. Every proposal and execution is logged with evidence and approver.
• They will not make retention decisions for you. They'll show you that 27 indices have no lifecycle policy and draft one; how long your compliance team needs those logs kept is a human call, and it stays one.

From cluster hygiene to standing habit

Across this series you've seen what degrades an Elasticsearch cluster, how to audit and fix each issue with native tools, and now how to make that discipline continuous. The gap between teams whose clusters stay green and teams who normalize yellow is rarely knowledge — it's cadence, and cadence is an automation problem. Teams that move from reactive firefighting to continuous, approval-gated index management typically cut search latency incidents and reclaim 20–40% of cluster storage within the first months, mostly from rollover and retention fixes that were always one stuck ILM step away.

Try CloudThinker free — 100 premium credits, no card required — and follow the Elasticsearch connection guide to see your own cluster's first findings within the hour.