Rollbar Automation Starts With Triage: 5 Signals Worth a Human

Open your Rollbar project and look at the number next to Active on the Items page. At a lot of mid-market SaaS companies it reads something like 4,000. Somewhere in those 4,000 unresolved items, filed between a five-year-old TimeoutError and a bot-generated 404, was the one new item that predicted last week's outage. It fired at 16:20 on a Tuesday, eleven minutes after a deploy. Nobody looked. Three days later the same stack trace took down checkout.

That's the real starting point for Rollbar automation. Before you write a notification rule or wire up an agent, you need to know which of Rollbar's signals deserve a human. Rollbar captures everything, and a team that treats every item equally ends up treating none of them at all.

This guide walks through the five triage failure modes we see most often: how each one happens, and one concrete way to detect it in your own project today. For the end-to-end picture before diving in, start with CloudThinker × Rollbar in three steps.

Part two is a hands-on guide to error-response automation with Rollbar's native tools: notification rules, deploy tracking, RQL and webhooks. Part three covers putting an AI agent on triage duty.

01Inbox

Graveyard

Thousands of Active items, none owned, most last seen months ago.

02Routing

One channel

New, reactivated and spiking items flattened into a single feed.

03Grouping

Fingerprints lie

One bug split into fifty items, or fifty bugs hidden in one.

04Correlation

No deploys

Nothing reports releases, so every regression hunt starts from zero.

05Silence

Mute culture

Muting becomes the default answer, and the real error rate moves inside it.

The five triage failure modes this guide covers, each with a check you can run today.

1. The item inbox that became a graveyard

What it is. Rollbar's Items page is designed as an inbox. New items arrive as Active, a person resolves or mutes them, and the list stays short. In practice most projects hit an inflection point, usually a few months after adoption, where items arrive faster than anyone processes them. Past that point the inbox stops being a queue and becomes a graveyard: thousands of Active items, none owned, none assigned, all technically "open."

Why it happens. Resolving an item needs a decision (is this fixed, worth fixing, or noise?), and decisions need an owner. Rollbar supports item assignment, but few teams use it, so every item is everyone's job, which means it's no one's. Once the count clears a few hundred, the page loses its signal value and engineers stop opening it except during incidents.

How to detect it. Two checks. In the UI: Items, status filter Active, sort by Last occurrence. If page one is full of items last seen months ago, you have a graveyard, not a queue. Then quantify it with RQL (left sidebar, RQL): count how many distinct items actually occurred in the last 30 days and compare that with your total Active count.

SELECT item.counter, count(*) AS occurrences
FROM item_occurrence
WHERE timestamp > unix_timestamp() - 2592000
GROUP BY item.counter
ORDER BY occurrences DESC

If that query returns 300 items but your Active filter shows 4,000, roughly 3,700 items are pure backlog. Decide once: bulk-resolve everything older than a quarter. Anything real will reactivate on its next occurrence, and that reactivation is a better signal than the stale entry ever was.

Typical cost. Teams in this state routinely spend 3 to 5 engineer-hours a week on ad-hoc "what is this error" archaeology, almost all of it during or after incidents, when it's most expensive.

2. New, reactivated and spiking: three signals, one attention budget

What it is. Rollbar emits three different signals that most teams flatten into one Slack channel: a new item (a stack trace never seen before), a reactivated item (a resolved item that came back), and an occurrence spike (a known item suddenly firing at 50 times its baseline). They don't deserve equal attention.

01Regression

New item

A stack trace never seen before, usually minutes after the deploy that caused it.

02Failed fix

Reactivated

A resolved item came back. The fix was incomplete or old code is running again.

03Environment

Spike

A known item firing far above its baseline. Usually a dependency, retry storm or bad actor.

Three signals, three different meanings. Only the first two reliably mean a human should look now.

A new item after a deploy is the highest-value signal Rollbar produces. It's a regression announcing itself, usually within minutes of the code that caused it.
A reactivated item means either the fix didn't work or a rollback or partial deploy brought old code back. Both deserve a person, now.
An occurrence spike on a long-known item is usually an environmental symptom (a dependency down, a retry storm, a bad actor), not a new bug. It matters, but it's an investigation signal, not a fix signal.

Why it happens. The default instinct is to route "all errors" to one channel with one severity. Within weeks the channel is 95% occurrence noise from known items, and the two signals that almost always mean "a person should look now" drown in it.

How to detect it. Check whether your project even distinguishes them: Settings, Notifications, then Slack (or your channel), and read the trigger list. Rollbar supports separate rules for New Item, Reactivated Item, 10^nth Occurrence and High Occurrence Rate (notifications docs). If you have one rule with the trigger "Every occurrence", or a New Item rule with no environment filter (so staging noise pages production engineers), this failure mode is live in your project.

Here's what a sane split looks like. The fix, per-trigger and per-environment rules, is the core of part two.

Signal	What it usually means	Where it should go
New item, production, level error or above	A regression from a recent deploy	Team channel, read the same hour
Reactivated item	A fix that didn't hold	Team channel, higher urgency
High occurrence rate	A release on fire, or a dependency down	On-call pager
Every occurrence	Raw stream	A machine (webhook), never a person

Typical cost. This is alert-fatigue math. When 19 of 20 pings are known-item noise, response time to the twentieth, the real one, stretches from minutes to days.

3. Grouping gone wrong: one bug as fifty items, fifty bugs as one

What it is. Rollbar groups occurrences into items by fingerprinting the stack trace (grouping docs). When fingerprinting misfires, triage breaks in one of two directions.

Over-splitting: one underlying bug shows up as fifty near-identical items. It's common when a dynamic value (an ID, a memory address, a generated class name) leaks into the message or trace, so every occurrence looks unique. You resolve one item and the "same" error stays active under forty-nine other counters.

Over-grouping: fifty different failures share one generic fingerprint. The classic case is a top-level Error or a framework's catch-all handler. One item quietly piles up six digits of occurrences spanning a dozen root causes, and resolving it means nothing.

Why it happens. Default fingerprinting is a heuristic tuned for typical stack traces. Minified JavaScript without source maps, wrapped exceptions and log-style messages with interpolated values all defeat it. Nobody notices because grouping failures don't alert. They just wear away the meaning of the item list.

How to detect it. Over-splitting: on the Items page, search for a distinctive phrase from a known error's title and count how many separate active items match. More than a handful for one code path means split fingerprints. Over-grouping: sort Items by Occurrences and open the top item's Occurrences tab. If the individual traces show different exception classes or different innermost frames, that item is a bucket, not a bug. You can also hunt for catch-all classes in RQL:

SELECT body.trace.exception.class, count(*) AS occurrences
FROM item_occurrence
WHERE timestamp > unix_timestamp() - 604800
GROUP BY body.trace.exception.class
ORDER BY occurrences DESC

A generic class (Error, RuntimeException) dominating that list is your cue to add custom grouping rules or merge the split items in the UI. For browser projects, upload source maps first. Many "split" items are the same minified frame at different offsets.

Typical cost. It varies a lot, but over-grouping is the dangerous direction. A new bug landing inside a muted catch-all item is invisible until customers report it.

4. Deploy tracking: the correlation signal almost nobody wires up

What it is. Most production errors come from deploys, and Rollbar can tell you which deploy, automatically, if you report deploys to it. With deploy tracking on, every item shows the last deploy before its first occurrence, the Versions view shows which code version introduced or resolved which items, and "did the release break something?" becomes a lookup instead of a forensic exercise. In our experience it's the most underused feature in the product.

Why it happens. Deploy tracking needs one extra step in the CI pipeline, a POST to the deploy API with the revision SHA. Error reporting works fine without it, so the step never gets prioritized. The team then pays for the omission in every investigation, cross-referencing item timestamps against the deploy log by hand.

How to detect it. Thirty seconds: open your project's Deploys page in the left sidebar. If it's empty, or the last recorded deploy came from an engineer's laptop eight months ago, you're running blind. Check the Versions view too. With code_version set on occurrences, you get regression detection nearly for free. A quick RQL check tells you whether your SDKs send it at all:

SELECT code_version, count(*) AS occurrences
FROM item_occurrence
WHERE timestamp > unix_timestamp() - 86400
GROUP BY code_version
ORDER BY occurrences DESC

One row with an empty value means no service reports a version. Part two shows the one-line SDK change and the CI call that fix it.

Typical cost. Deploy-correlated investigation typically turns a 30 to 60 minute "when did this start and what shipped around then" hunt into a two-minute confirmation. Multiply that by every incident this quarter.

5. Error budgets versus mute-everything culture

What it is. Faced with the noise from failure modes 1 to 4, teams reach for Mute. Muting is legitimate for items nobody can act on, like third-party script errors or bot traffic. But muting spreads. Once it's the normal response to annoyance, the muted set only grows, and eventually the project's real error rate lives mostly inside it.

The opposite discipline is an error-budget mindset: decide how much error volume per service is acceptable, watch the trend, and treat sustained budget burn as work to schedule rather than sound to silence.

Why it happens. Mute is instant relief with an invisible cost. Few teams have a "review muted items" ritual, so mutes are forever, and a muted item that later grows tenfold stays silent.

How to detect it. On the Items page, set the status filter to Muted and compare its count with your Active count. A muted set that rivals Active, or contains high-occurrence items nobody can justify, means muting has become policy rather than exception. Skim the top ten muted items by occurrences and ask one question of each: would we want to know if this doubled? For most, the honest answer is yes.

Typical cost. This is where the outage-in-the-inbox story from the top usually lives. The predictive item wasn't unseen. It was muted six months earlier by someone who has since left.

Why triage by heroics doesn't scale

None of the five problems is exotic, and each check takes minutes. The structural reason they keep coming back is that Rollbar converts code paths into items automatically, and converting items into decisions takes human attention that nobody has budgeted.

01Month 0

Inbox

New items get read, resolved or muted the same week.

02Month 3

Backlog

Arrivals outpace decisions. Nobody owns the leftovers.

03Month 6

Graveyard

The Items page is opened only during incidents.

04After an outage

Cleanup

A burst of bulk-resolving, then the slide starts again.

Triage decays at the rate items arrive. A quarterly cleanup can't keep up with a daily stream.

Three things make the decay hard to stop with process alone:

Every rule is static. A notification threshold or grouping rule is right for the traffic and code you had when you wrote it. The next refactor or traffic doubling makes it wrong, silently.
Context lives in four places. The stack trace is in Rollbar, the diff is in Git, the deploy log is in CI, and the infrastructure state is in your cloud console. Every triage decision means joining them by hand.
Knowledge leaves with people. Why an item was muted, which items are "the same bug", which service owns which frame: that lives in heads, not in the project.

So triage happens in bursts (after an incident, before an audit) and slides back to graveyard state within a quarter.

Where this series goes next

The durable fix starts with structure: route the three signal types differently, fix the fingerprints that lie, wire deploy tracking into CI, and put a floor under the mute button. Rollbar's native tooling can do a surprising amount of this (notification rules, the deploy API, RQL, resolve-in-version), and part two builds that setup step by step with exact API calls.

What native tooling can't do is read the trace and decide. Part three shows how CloudThinker connects to Rollbar read-only, picks up new items, reactivations and spikes as they happen, correlates each with the deploy that introduced it, and proposes a diagnosis with evidence. Every action beyond investigation waits for your approval. For a single item followed from first occurrence to verified fix, read the Rollbar full-lifecycle deep dive.

Try CloudThinker free with 100 premium credits and no card required, or continue with the native-tools automation guide.