How To

How to Automate Incident Response with Better Stack's Native Tools

Part two of the Better Stack series: tune monitors, heartbeats, escalation policies, the incidents API and webhooks, Telemetry logs and status pages, with exact settings and curl calls. It ends on the ceiling: native tooling cuts time to human, but time to cause is still manual.

8 min read
Cover Image for How to Automate Incident Response with Better Stack's Native Tools

How to Automate Incident Response with Better Stack's Native Tools

Your escalation policy already works. A monitor fails, the incident opens, the on-call engineer's phone rings inside two minutes, and they acknowledge before the second escalation step fires. Your Better Stack integration did exactly what it was configured to do. Then the automation stops, and the engineer starts the same manual investigation they ran last month.

This article pushes that boundary as far as it goes with native features alone. Part one mapped where on-call time actually leaks: the 3 a.m. context hunt, cron jobs nobody wired to a heartbeat, log search in a separate tab, timelines rebuilt by hand, status pages updated last. Here we work feature by feature, with exact settings and copy-pasteable calls.

01Detect

Uptime monitors

Frequency, regions and confirmation period decide what counts as down.

02Detect

Heartbeats

Cron jobs and workers check in, or an incident opens.

03Route

Escalation policies

Whoever is on call, by severity, with timed steps.

04Extend

API and webhooks

Script incidents and run your own receivers on every state change.

05Evidence

Telemetry logs

Search and alert on logs in the same product as the incident.

06Communicate

Status pages

Components flip with the linked monitor, no human needed.

The six native features this guide configures, in the order that pays off.

Budget an afternoon. The payoff is fewer false pages and much richer context when a real one lands. All API examples use the Uptime API v2 with a token from Uptime → Integrations → API tokens (API docs):

export BETTERSTACK_TOKEN="your-api-token"

1. Uptime monitors: frequency, regions, confirmation period

Escalations, webhooks and status pages all key off monitors, so tune these first. Four settings do most of the work:

  • check_frequency: 30 seconds on paid plans (the free tier checks every 3 minutes). For anything customer-facing, 30 seconds is the difference between catching a bad deploy and hearing about it from customers.
  • regions: check from all four (us, eu, as, au). Better Stack re-verifies a failure from another location before opening an incident, so one PoP with a bad route becomes a non-event instead of a page.
  • confirmation_period: how long the check must keep failing before anyone is alerted. 60 to 120 seconds absorbs restarts and blips. Zero means every flap pages someone.
  • recovery_period: how long the check must stay green before the incident auto-resolves. Keep it around 180 seconds, or a flapping service will "resolve" itself mid-outage.

Create one through the API:

curl -X POST https://uptime.betterstack.com/api/v2/monitors \
  -H "Authorization: Bearer $BETTERSTACK_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "monitor_type": "expected_status_code",
    "url": "https://api.example.com/health",
    "expected_status_codes": [200],
    "check_frequency": 30,
    "regions": ["us", "eu", "as", "au"],
    "confirmation_period": 120,
    "recovery_period": 180,
    "request_timeout": 15
  }'

Prefer expected_status_code or keyword monitors over the plain status type for health endpoints. A 302 to your login page is "up" to a status check and very much down to your users.

Maintenance windows fix the deploy-night page. Set them per monitor (Monitor → Configure → Maintenance window, or maintenance_days, maintenance_from, maintenance_to and maintenance_timezone in the same payload) so the Tuesday 01:00 to 02:00 UTC deploy never opens an incident. If your team silences monitors by pausing them instead, that's a standing risk: paused monitors don't come back on their own.

2. Heartbeats: cover the things that don't listen on a port

Uptime monitors probe from outside. Heartbeats flip the direction for cron jobs, queue workers and backups: the job must check in, or Better Stack opens an incident. Part one found these are the most commonly unwired class of failure. Wiring one takes two steps.

Create the heartbeat (period is the expected interval and grace the slack before alerting, both in seconds):

curl -X POST https://uptime.betterstack.com/api/v2/heartbeats \
  -H "Authorization: Bearer $BETTERSTACK_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "nightly-db-backup", "period": 86400, "grace": 3600}'

The response includes a unique heartbeat URL. Add it to the job itself so the ping only fires on success:

0 2 * * * pg_dump mydb | gzip > /backups/mydb.sql.gz \
  && curl -fsS https://uptime.betterstack.com/api/v1/heartbeat/YOUR_TOKEN

You can also report a failure right away, without waiting out the grace period, by calling the same URL with /fail appended. Inventory rule: every cron job and queue consumer that matters gets a heartbeat, and the heartbeat call lives in the job's own definition, not in a wiki.

3. Escalation policies and on-call calendars

Set up the on-call calendar first (Uptime → On-call & escalations): a rotation with explicit handoff times, plus overrides for holidays. Then build escalation policies that point at whoever is on call rather than named people. A shape that holds up in practice:

01Immediately

On-call

Push and phone call to whoever the calendar says is on call.

02After 5 min

Team

Still unacknowledged, so the whole team is notified.

03After 15 min

Lead

The engineering lead gets the call.

Reference people by rotation, not by name. Named humans go on vacation.

Use calls for the first step. Push notifications alone don't survive a misconfigured do-not-disturb. Then attach different policies to different monitor groups: checkout API failures deserve the phone call, the staging environment deserves an email. Audit what you have:

curl "https://uptime.betterstack.com/api/v3/policies" \
  -H "Authorization: Bearer $BETTERSTACK_TOKEN"

If that returns one default policy attached to everything, severity routing is the highest-leverage fix in this whole article. Undifferentiated paging is how teams train themselves to ignore pages.

4. The incidents API and incident webhooks

Better Stack records every incident with its cause, timestamps and acknowledgment state, and that data is scriptable. Pull last week's incidents for a review:

curl "https://uptime.betterstack.com/api/v2/incidents?from=2026-09-28&to=2026-10-05" \
  -H "Authorization: Bearer $BETTERSTACK_TOKEN"

Acknowledge or resolve programmatically, which is useful when your deploy tooling knows a rollback just fixed the problem:

curl -X POST "https://uptime.betterstack.com/api/v2/incidents/INCIDENT_ID/resolve" \
  -H "Authorization: Bearer $BETTERSTACK_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"resolved_by": "deploy-bot@example.com"}'

Outgoing webhooks (Uptime → Integrations) are the extension point: Better Stack POSTs JSON to your endpoint when an incident starts, is acknowledged or resolves. Teams use this to open a ticket, post an enriched Slack message with runbook links, or run a first-response script (restart the service, capture a thread dump) before anyone is awake. It's real automation with one catch we'll come back to: you write, host and maintain every receiver yourself.

5. Better Stack Logs: search and alert next to your incidents

The "logs in a separate tab" problem from part one has a partial native fix: ship logs into Better Stack Telemetry so the incident and the evidence live in one product (Logs docs). Create a source (Telemetry → Sources → Connect source), then ship with Vector, the OpenTelemetry Collector, or plain HTTP using the source token and the ingesting host shown on the source's page:

curl -X POST "https://$INGESTING_HOST" \
  -H "Authorization: Bearer $SOURCE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"message": "checkout failed: connection pool exhausted", "level": "error", "service": "api"}'

Two features earn their keep during incidents. Live tail with filtering gives grep-speed search across services while the incident is open. Log-based alerts page on what monitors can't see, like a spike of 5xx responses in access logs or any occurrence of connection pool exhausted, and route through the same escalation policies as uptime checks. An HTTP monitor confirms the symptom. A log alert often points at where the cause is, which is a better page to wake up to.

6. Status pages that update themselves

Wire the status page to monitors so the component state changes without anyone touching it mid-incident:

# Create the page
curl -X POST https://uptime.betterstack.com/api/v2/status-pages \
  -H "Authorization: Bearer $BETTERSTACK_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"company_name": "Example", "subdomain": "status-example", "timezone": "UTC"}'

# Attach a monitor as a public resource
curl -X POST https://uptime.betterstack.com/api/v2/status-pages/PAGE_ID/resources \
  -H "Authorization: Bearer $BETTERSTACK_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"resource_id": MONITOR_ID, "resource_type": "Monitor", "public_name": "API"}'

When a linked monitor's incident opens, the component flips. When it recovers, it flips back. Human-written updates are then reserved for what humans are needed for, explaining impact and ETA, instead of the mechanical "we are aware" post.


What native automation covers

Part one gapNative fixWhat's still manual
Context huntMonitor naming, runbook links in webhook messagesFinding which service and which change
Unwired heartbeatsHeartbeats with /fail, wired in the job definitionNoticing the jobs nobody enrolled
Logs in another tabTelemetry sources, live tail, log alertsWriting the right query for this incident
Hand-built timelinesIncident lifecycle via the APIThe investigation itself, in Slack and scrollback
Silent status pageMonitor-linked componentsThe words: impact and ETA

The ceiling: escalation finds a human, not a cause

Configure everything above and you have a good setup: false pages mostly gone, silent cron failures covered, severity-routed escalation, logs one click from the incident, a status page that keeps itself honest. Be just as clear about what you don't have.

01Solved

Time to human

Checks confirm in seconds and escalation reaches someone in minutes.

02Still manual

Time to cause

Which service, what changed, which log line. Untouched by any setting here.

03Yours to maintain

Webhook code

Every receiver is a script your team writes, hosts and updates.

Native tooling optimizes the minutes before acknowledgment. The larger cost comes after.

Detection stops at the boundary Better Stack can see. It observes the symptom: the 500, the timeout, the missed heartbeat. The cause lives on the other side, in the disk that filled, the pod the OOM killer took, the connection pool a bad deploy exhausted. No monitor setting reaches there.

Investigation is manual, every time. The escalation policy's job ends when a human acknowledges. Which monitor, which service, what changed in the last hour, which log line matters: the 20 to 40 minute sequence from part one is untouched by everything configured here.

Webhook automation is code you own. Every receiver is a script your team writes, hosts, secures and updates when the payload or the infrastructure changes. Most teams build two, then stop. And a receiver runs the same steps whatever the incident is, because it was written before the incident happened.

Native tooling optimizes time to human. The larger cost, time to cause, still comes out of an engineer's night.

What comes next

Part three automates the investigation itself. CloudThinker connects to Better Stack via OAuth in about two minutes, read-only by default, picks up incidents as they open, pulls the failing monitor's context and recent logs, and checks the infrastructure behind the symptom, so the on-call engineer arrives to a diagnosis instead of a blank timeline, with the escalation chain untouched.

Related reading

See what it finds in your own stack: Try CloudThinker free with 100 premium credits and no card required.

betterstackincidentresponseuptimeoncallmonitoringsre