Automating MongoDB Performance Tuning with an AI Database Agent

Every MongoDB index recommendation has an expiry date. The compound index that made your dashboard query instant in January starts limping in June, because a product launch changed the query shape and now the equality field sits behind the sort field. The index review you ran last quarter said "drop these five, they're unused," but one of them quietly became load-bearing when a new report shipped. Index advice is only correct relative to a workload, and the workload never stops moving.

This is part three of our MongoDB performance series. Part one mapped the five patterns that degrade MongoDB performance: collection scans, wrong compound-index order, unbounded array growth, working sets that outgrow RAM, and write-side drag. Part two walked through auditing all of them with native tools: the profiler, explain("executionStats"), $indexStats, mongotop and mongostat.

That audit gives you a solid picture of your cluster for the week you ran it. This article covers the continuous version: how CloudThinker keeps that audit running against the live workload, what one investigation looks like end to end, what a first pass on a typical deployment surfaces, and what it won't do without your explicit approval.

---

Why the quarterly audit can't keep up

The part-two audit is correct. Its limits are about time, and they're structural.

• The profiler only sees what crosses slowms. A query shape that drifts from 30 ms to 90 ms over three weeks never shows up at a 100 ms threshold, yet at 9,000 calls an hour it's one of your biggest costs.
• $indexStats forgets. Counters reset on every restart and are per member. "Zero ops" after a Sunday patch window means nothing, and a reviewer who checks only the primary misses the reporting reads on a secondary.
• Every finding becomes a ticket. Build in a quiet window, verify with explain, hide the old index, wait, drop. Each step needs someone to remember it, so the follow-through rate drops with each step.
• Correct decisions age. An index that was right for January's query mix is wrong for June's. A point-in-time audit can't see that, because it was right when it ran.

The expertise to do this well exists on your team. What's missing is someone running it every day.

Connecting CloudThinker: read-only, about five minutes

01Database

MongoDB

• Profiler and server logSlow operations by query shape
• explain and $indexStatsPlans, index usage per member
• serverStatusWiredTiger cache, eviction, connections

read + clusterMonitor

02Investigation

CloudThinker

• Track query shapesFrequency, ratio and latency trend
• DiagnosePlan, index fit, cache or replication
• Propose the changeESR index, hide-then-drop, with rollback

No agent on your servers

03Where the team works

Chat and tickets

• SlackFindings and approvals in the thread
• Microsoft TeamsSame flow for Teams-first orgs
• JiraSchema follow-ups with evidence linked

Nothing new to watch

CloudThinker connects to MongoDB the way you'd want any third party to: with a dedicated user holding read-only roles. That means read on the databases you scope it to, plus clusterMonitor so it can see serverStatus, $indexStats and profiler output. No driver changes, no agent on your servers, no write access at connection time. You create the user, paste the connection string, and the first scan starts. The MongoDB connection guide lists the exact roles so your security review can verify what's granted and what isn't.

db.getSiblingDB('admin').createUser({
  user: 'cloudthinker_ro',
  pwd: passwordPrompt(),
  roles: [
    { role: 'read', db: 'orders' },
    { role: 'clusterMonitor', db: 'admin' },
  ],
});

It works the same whether you run a self-managed replica set or Atlas, because it reads the database's own instrumentation, not a vendor API.

What gets watched continuously

The part-two audit, re-run continuously instead of on the afternoon you could spare for it:

• Profiler output, aggregated over time. Slow operations clustered by query shape, with each shape's frequency, docsExamined-to-nReturned ratio and latency trend tracked. A query that regressed 40% over three weeks is a finding, even if it never crossed your slowms threshold on any single day.
• Explain plans on the shapes that matter. For the top recurring shapes, the winning plan gets inspected: COLLSCAN stages, in-memory SORT stages, and indexes used for only a prefix of the predicate.
• Index health through $indexStats, on every member. Indexes with zero or near-zero accesses across a full business cycle, redundant prefixes (a { userId: 1 } index shadowed by { userId: 1, createdAt: -1 }), and the write amplification and cache footprint they cost. Counter resets get tracked, so a restart doesn't make a busy index look unused.
• Schema shape drift. Document size percentiles per collection, flagging the unbounded-array pattern from part one before a hot document approaches the 16 MB cap. That failure mode never shows up in an index review because it's a modeling problem, not an index problem.
• Server vitals from serverStatus. WiredTiger cache eviction pressure, dirty-cache percentage and connection churn: the signals that tell you whether slow queries are a query problem or a working-set problem.

The value is cadence, not cleverness. The compound index that goes stale in June gets flagged in June, with before and after explain plans attached, not discovered in the Q3 audit.

One regression, end to end

01Signal

Shape regresses

A query shape gets slower or more frequent, even below slowms.

02Diagnose

Read the plan

explain, index fit, cache counters and lag, in that order.

03Propose

One change

An ESR index, a hide, or a schema note, with projected effect.

04Verify

Before and after

Same explain re-run, keys examined compared, regression watch set.

Illustrative numbers. After a release adds a "recent open orders" panel for each tenant, p95 on the orders API climbs from 15 ms to 600 ms over two days. No query errors.

1. Name the readings. Four explanations fit, and each predicts something different:

2. Read the plan. The new shape filters { tenantId, status: "open" } and sorts by createdAt: -1. The planner picks the existing { createdAt: 1, tenantId: 1 } index, walks the whole date range and filters tenants as it goes, then sorts in memory.

3. Propose one change. An ESR-ordered index that serves the shape, with the evidence and the plan for verifying it:

db.orders.createIndex({ tenantId: 1, status: 1, createdAt: -1 });

The proposal names a low-traffic window for the build (from the last two weeks of mongotop and opcounters), says replication lag will be watched while it runs, and projects keys examined dropping to roughly the 50 documents returned. It also notes that { createdAt: 1, tenantId: 1 } still serves two other shapes, so it isn't a drop candidate.

4. Verify. After the approved build, the same explain is re-run and compared, and the shape's p95 is watched for a week. If it doesn't land where projected, that's a new finding, not a closed ticket.

Proposals come with evidence and a rollback path

When CloudThinker proposes a change, it looks like the case a careful DBA would build:

• Index proposals follow the ESR rule: equality fields first, then sort, then range. They cite the query shapes the index serves, their frequency and the projected plan change.
• Drop proposals are staged, not abrupt. For an unused index, the proposal is to hide it first with hideIndex(), watch for regressions over an observation window you set, and only then drop. Hiding is instantly reversible; a dropped 40 GB index is a long rebuild.
• Schema-shape findings are advisory. You'll see that the events array in your sessions collection grows without bound, with a sketch of the bucketing alternative. Restructuring documents is an engineering decision CloudThinker won't make for you.

Nothing changes outside the limits you set

CloudThinker acts within Auto Mode, so you decide what runs on its own and what waits for approval. Every finding, proposal, approval and execution lands in an audit trail with its evidence.

What a first pass typically surfaces

Illustrative numbers: a composite of first scans on a mid-market deployment, a three-node replica set with ~400 GB of data and mixed OLTP and reporting. Yours will differ.

Look at the shape of the list: two of the six findings (the misordered compound index and the shadowed prefix) are things a point-in-time audit plausibly created, correct decisions that aged out. That's the argument for watching continuously rather than auditing quarterly.

Why this beats the traditional setup

The deeper difference is that index advice stops being a document and becomes a standing loop. The workload moves every week; the checks now move with it.

Getting started

1. Connect read-only and ask where the time goes.

"Which query shapes consumed the most cumulative execution time this week, and which of them are doing collection scans? Propose only, change nothing."

2. Find the dead weight.

"List every index with zero reads in the last 30 days across all replica set members, what each costs in RAM and write overhead, and a staged hide-then-drop plan."

3. Work one slow query properly.

"Propose an index for the slow orders query and show me the explain output before and after, plus which existing indexes it would make redundant."

4. Hand off the routine cleanup.

"In staging, hide every index with zero accesses on every member for 30 days and report what you hid. Don't build or drop anything."

What CloudThinker won't do

The questions you should ask any tool holding a connection string to production, answered plainly:

• Read-only by default. The connection user has no write roles. Executing anything requires you to grant scoped permissions on purpose, and every change goes through Auto Mode.
• No abrupt index drops. Drops are staged through hidden indexes first, so a mistake is one unhideIndex() away from undone.
• No query rewrites or document restructuring. Schema-shape findings arrive as evidence and options; the modeling decision stays with your team.
• No actions outside the audit trail. Every proposal and execution is logged with its evidence and its approver.

Related reading

• Part one: 5 ways fast queries turn slow at scale
• Part two: the profiler, explain, and $indexStats
• Auto Mode
• CloudThinker Connections
• Automating PostgreSQL performance tuning

From snapshot to standing watch

You know what bad looks like in MongoDB, and you know how to find it with the profiler and explain. The gap is that the workload shifts weekly and your audit calendar doesn't. Moving from quarterly index reviews to a continuous loop, with a person approving every change, closes the two expensive gaps: the COLLSCAN that ships silently with a new feature, and the "safe to drop" index that no longer is.

Try CloudThinker free with 100 premium credits and no card required, then follow the MongoDB connection guide and see your own first-findings table within the hour.