Product

Automating Neon Postgres Tuning with an AI Database Agent

On Neon, an index proposal can arrive already proven. CloudThinker watches query stats, autoscaling headroom and connections, then rehearses each fix on a branch of production and measures it before asking for approval. Nothing runs DDL on main without a named approver.

10 min read
Cover Image for Automating Neon Postgres Tuning with an AI Database Agent

Automating Neon Postgres Tuning with an AI Database Agent

There are two kinds of index recommendations. The first is a line in a report: "consider adding an index on orders(customer_id)." Somebody still has to check it isn't redundant, guess its write cost, test it somewhere, and hope production behaves like staging. The second kind arrives already proven. The index was created on a copy-on-write branch of your actual production data ten minutes ago, the offending query re-run against it, and the proposal shows 3,400 ms to 45 ms measured on your rows, with the DDL waiting for your approval before it goes anywhere near main.

Neon Postgres is one of the few databases where the second kind is cheap enough to be the default. Branching gives you a disposable clone of production in seconds, which turns tuning from prediction into measurement. The catch is that nobody has time to run that loop by hand every day.

Part one mapped what performance means on serverless Postgres: cold starts, autoscaling ceilings, pooled versus direct connections, and how compute-hours pricing turns slow queries into money. Part two ran the DIY audit with native tools: the console Monitoring dashboard, pg_stat_statements, EXPLAIN (ANALYZE, BUFFERS), and branching as a manual test bench. This part is about making that loop continuous: CloudThinker watching the same signals every day and rehearsing every fix on a branch before it touches main.


Why Neon changes the automation math

Two properties of Neon make continuous tuning more valuable here than on a conventional Postgres host.

First, the bill is a live feedback signal. Neon charges for compute-hours, so a query that pins your endpoint at its autoscaling ceiling for an hour a day is a visible line item, and a minimum compute size left at 2 CU on a project that idles at 0.3 CU is money spent on nothing. On Neon, query tuning and cost tuning are the same activity, and both drift weekly as workloads change.

Second, validation is nearly free. On RDS you'd restore a snapshot to test an index against production-scale data. That's slow and expensive, so almost nobody does it. On Neon, a branch is a copy-on-write clone that exists in seconds and costs only the compute you run against it. Part two showed the manual branch, test, measure, apply workflow. The gap between knowing it and running it for every finding, every week, is the gap an agent fills.

Why a scheduled script doesn't close it

The obvious DIY move after part two is a cron job: snapshot pg_stat_statements, alert when a query's mean time doubles, page when CPU sits at max. That covers detection. It doesn't cover what happens next, and the next step is where the hours go.

When the endpoint pins at 4 CU, the right answer could be "raise the max," "fix this one scan," "the cache is too small for the working set," or "a dev branch is replaying production traffic." Each reading needs a different check, and the next check depends on what the last one found. A script follows the branches you wrote. It can't decide that the evidence points at a query rather than the sizing, then go create a branch, try an index and measure it.

Connecting: OAuth, about two minutes

01Database

NeonNeon

  • PostgreSQLStatistics viewspg_stat_statements, pg_stat_user_tables
  • Compute configurationAutoscaling range, scale-to-zero, endpoints
  • BranchesDisposable copies for rehearsal

OAuth, read-only by default

02Investigation

CloudThinkerCloudThinker

  • Track queries and headroomCaptured before each suspend resets them
  • Rehearse fixes on a branchMeasured before and after on real rows
  • Propose the changeCONCURRENTLY DDL, write cost, rollback

No DDL on main without approval

03Where the team works

Chat and tickets

  • SlackSlack or TeamsFindings and approvals in the thread
  • JiraFollow-ups with the branch results linked

Nothing new to watch

Authorize once, pick the projects it may see, revoke in one click from Neon's account settings.

CloudThinker connects to Neon through the OAuth flow: you click connect, authorize CloudThinker against your Neon account, pick the projects it may see, and the first analysis starts. No connection strings pasted into a form, no manually created database roles, and access is read-only by default. CloudThinker can query statistics views and read compute and branch configuration, but nothing at connection time grants it the right to change your schema or endpoints. Revoking access is one click in Neon's account settings. The Neon connection guide walks through it screen by screen, including exactly what the authorization does and doesn't allow.

If pg_stat_statements isn't enabled on the database yet, the first pass flags that. It's a single CREATE EXTENSION on Neon, covered in part two, and it's the highest-value prerequisite for everything that follows.

What CloudThinker watches on a Neon project

Once connected, it runs the part-two audit as a standing process rather than a quarterly afternoon:

  • Slow query trends. The top of pg_stat_statements by total and mean time, captured over time so a suspend doesn't erase the history. A query that regresses after a deploy is flagged as a regression with a date, not rediscovered three invoices later.
  • Autoscaling headroom. How often the endpoint sits at its maximum compute size, and for how long. Those windows are correlated with the specific queries driving them, so the answer might be "raise the max" or might be "fix this one scan."
  • The opposite failure: paying for a floor. Minimum compute sizes and scale-to-zero settings that don't match how each branch is actually used: the dev branch billing for 2 CU around the clock, or the production endpoint suspending overnight and serving the first customer a multi-second cold start.
  • Connection saturation. Applications on the direct endpoint that should be on the pooled one, connection counts creeping toward the limit, and the deploy-time spike pattern that comes before intermittent connection errors.
  • Index health. Unused indexes taxing every write, duplicate indexes, and missing ones implied by sequential-scan-heavy access patterns. The unindexed foreign key on a large child table remains the classic.

None of these checks is exotic; part two showed you can run each one yourself. What changes is cadence and memory: the dashboard you checked once becomes a signal that checks itself, with the before and after comparison attached when something moves.

One finding, rehearsed end to end

01Signal

Query moves

A query regresses against its baseline, or the endpoint pins at max CU.

02Hypotheses

Investigate

Plan, statistics, cache or sizing, each checked against the evidence.

03Rehearsal

Branch test

The candidate fix applied to a branch of main and measured on real rows.

04Decision

Approval

A named person approves the exact DDL for main, or it waits.

The proposal you review is an experiment that already ran, not a hypothesis you now have to test.

Here's a case from the findings table further down, walked through. The numbers are illustrative.

The signal. The production endpoint has spent about 90 minutes a day at its 4 CU maximum for two weeks, always between 09:00 and 11:00.

The hypotheses. Three readings fit "pinned at max":

  • Real load growth. Predicts broad increases in calls across many queries, and the fix is a higher max.
  • A cache that's too small. Predicts the LFC hit rate dropping well under 99% in the pinned windows, with reads spread across many queries.
  • One or two expensive queries. Predicts a few queries dominating total_exec_time inside those windows.

The check. Comparing the captured query snapshots for the pinned windows against the quiet hours answers it: total calls are flat week over week, the LFC hit rate dips only modestly, and two reporting queries account for most of the execution time between 09:00 and 11:00. Their plans show a Seq Scan on invoices filtering on account_id and issued_at, reading hundreds of thousands of buffers per call.

The rehearsal. CloudThinker creates a branch from main, runs both queries under EXPLAIN (ANALYZE, BUFFERS) for a baseline, creates the candidate index, runs ANALYZE, and runs them again:

-- on the rehearsal branch, never on main
CREATE INDEX idx_invoices_account_issued
  ON invoices (account_id, issued_at);
ANALYZE invoices;

Mean time for the slower query goes from 3,400 ms to 45 ms, and buffers read drop by two orders of magnitude. Then the branch is deleted.

The proposal. The finding in Slack says the ceiling isn't the problem and raising it would only buy more CU-hours for the same scans. It proposes the index for main as CREATE INDEX CONCURRENTLY, with both plans, the branch measurements, the expected write cost on invoices, and the rollback (DROP INDEX CONCURRENTLY). Nothing touches main until someone approves it.

Every proposal is rehearsed on a branch first

That walkthrough is the standard, not the exception. When CloudThinker proposes an index or a query rewrite, it doesn't reason from planner estimates alone. It creates a branch from main, applies the candidate change there, replays the affected queries against your real data distribution, and measures. The proposal you review contains:

  • the exact queries from pg_stat_statements that motivated it, with call counts and current total time;
  • the measured before and after timings from the branch run, real numbers from your rows rather than estimated cost units;
  • the EXPLAIN (ANALYZE, BUFFERS) comparison from both sides;
  • the write-side cost, because every index taxes INSERT and UPDATE, and a proposal on a hot table says so;
  • the DDL itself, written as CREATE INDEX CONCURRENTLY so applying it to main doesn't block writes.

The branch is disposable and cleaned up afterward, so the rehearsal costs a few minutes of small-compute time. "Drop this index" proposals get the same treatment in reverse: zero scans across the observation window, plus a branch-verified check that the queries you care about don't regress without it.

Nothing runs DDL on main without approval

CloudThinker acts within Auto Mode, so you decide what runs on its own and what waits for approval. One rule sits above that: no DDL executes on main without approval. Creating a rehearsal branch is a low-risk, reversible action; altering your production schema is not. Anything that changes main (index creation or removal, compute-size changes, endpoint settings) requires explicitly granted write access and a named person's decision. Every proposal, approval and execution lands in an audit trail with its evidence and its approver.

What a first week typically surfaces

The numbers below are illustrative: a composite of what a first analysis tends to return on a mid-market Neon setup with a production project around 60 GB, autoscaling 1 to 4 CU, and a handful of preview and dev branches. Yours will differ.

FindingDetailTypical impact
Morning cold startsProduction endpoint suspends overnight; first query of the day takes seconds instead of millisecondsDaily latency spike on a customer-facing path
Autoscaling ceilingEndpoint pinned at 4 CU for ~90 min a day, driven by two reporting queriesQueuing during peak; compute-hours cost
Dev branch floorDev branch minimum compute left at 2 CU, idle overnight and weekendsSteady spend for zero work
Direct-endpoint trafficApp connecting to the direct endpoint; spikes to ~85% of the connection limit during deploysIntermittent connection errors
Missing FK indexUnindexed foreign key on a 40M-row child table; parent deletes scan itMulti-second locks on delete paths
Unused indexes9 indexes, 11 GB total, zero scans in 60 daysWrite amplification and storage
Query regressionSearch endpoint 12x slower since a schema change three weeks agop95 breach nobody had dated

Nothing here is clever. The value is that the table refreshes itself daily, each row arrives with a branch-validated, approval-gated fix, and the regression gets a date attached the week it happens instead of during the next incident review.

Why this beats the traditional approach

Quarterly tuning passCron job and alertsCloudThinker on top
Query historyWhatever survived the last suspendSnapshots, if someone built themCaptured continuously
Pinned at maxNoticed at invoice timeAn alertA named cause: sizing, cache or a query
Fix validationStaging that approximates prodNoneMeasured on a branch of main
Cost and performanceSeparate reviewsSeparate alertsOne finding, both effects
Change to mainA ticket and a change windowA ticketA proposal with evidence, approved in chat, audited

The deeper difference is where the proof happens. Every other approach hands you a hypothesis and leaves the testing to you. Here the test has already run on your data before anyone is asked to decide.

Prompts to try in your first session

CloudThinker is conversational. You ask in plain language and get answers cited against pg_stat_statements and the project's configuration, so you can verify rather than trust:

"Show the top 10 queries by total execution time this week and flag anything that regressed versus the previous 30 days."

"How much time did the production endpoint spend at its autoscaling max last week, and which queries were running during those windows?"

"Create a branch, test an index for the slow search query on it, and show me the measured before and after. Don't touch main."

What CloudThinker won't do

"AI agent connected to production Postgres" should make you ask questions. Here are the answers:

  • It's read-only by default. The OAuth connection starts with observation. Changing anything requires you to explicitly grant write access, and every change to main then needs a named person's approval.
  • It never runs DDL on main without approval. Branch rehearsals are disposable; your production schema changes only when a named person approves that specific action.
  • It doesn't act outside the audit trail. Every proposal, branch experiment and execution is logged with its evidence and approver.
  • It doesn't replace judgment on architecture. It can measure that a table wants partitioning before it wants another index. Whether to partition is an engineering decision, and it stays yours.

From snapshot to standing process

What's left after part two is cadence, and cadence is an automation problem, not an effort problem. Teams that move from occasional tuning passes to continuous, branch-validated, approval-gated tuning typically stop paying for idle compute floors, catch regressions the week they ship, and keep the autoscaling ceiling for genuine load instead of unindexed scans.

Try CloudThinker free with 100 premium credits and no card required, and follow the Neon connection guide to see your own first findings table within the hour.

Related reading: what slows serverless Postgres down, Neon tuning with native tools, and Auto Mode.

neondatabaseperformancepostgresserverlesspostgresneonbranchingaiagentcloudthinker