Rollbar Automation Starts With Triage: 5 Signals Worth a Human

Open your Rollbar project and look at the number next to Active on the Items page. At a lot of mid-market SaaS companies it reads something like 4,000. Somewhere in those 4,000 unresolved items — filed between a five-year-old TimeoutError and a bot-generated 404 — was the one new item that predicted last week's outage. It fired, nobody looked, and three days later the same stack trace took down checkout.

That's the real starting point for Rollbar automation. Before you write a single notification rule or wire up an agent, you need to know which of Rollbar's signals actually deserve a human — because the tool captures everything, and a team that treats every item equally ends up treating none of them at all. This guide walks through the five triage failure modes we see most often, how each one happens, and one concrete way to detect it in your own project today.

This is part one of a three-part series. Part two is a hands-on guide to error-response automation with Rollbar's native tools — notification rules, deploy tracking, RQL, webhooks. Part three covers putting an AI agent on triage duty.

1. The item inbox that became a graveyard

What it is. Rollbar's Items page is designed as an inbox: new items arrive as Active, a human resolves or mutes them, the list stays short. In practice, most projects hit an inflection point — usually a few months after adoption — where items arrive faster than anyone processes them. Past that point the inbox stops being a queue and becomes a graveyard: thousands of Active items, none owned, none assigned, all technically "open."

Why it happens. Resolving an item requires a decision (is this fixed? worth fixing? noise?), and decisions need an owner. Rollbar supports item assignment, but almost nobody uses it, so every item is everyone's job — which means it's no one's. Once the count clears a few hundred, the page loses all signal value and engineers stop opening it except during incidents.

How to detect it. Two checks. In the UI: Items → status filter: Active, sort by Last occurrence. If page one is full of items last seen months ago, you have a graveyard, not a queue. Then quantify it with RQL (left sidebar → RQL): count how many distinct active items actually occurred in the last 30 days versus your total Active count.

SELECT item.counter, count(*) AS occurrences
FROM item_occurrence
WHERE timestamp > unix_timestamp() - 2592000
GROUP BY item.counter
ORDER BY occurrences DESC

If that query returns 300 items but your Active filter shows 4,000, roughly 3,700 items are pure backlog — decide once (bulk-resolve everything older than a quarter; anything real will reactivate) and never let the pile rebuild.

Typical cost. Teams in this state routinely spend 3–5 engineer-hours a week on ad-hoc "what is this error" archaeology, almost all of it during or after incidents when it's most expensive.

2. New vs. reactivated vs. spiking — three signals, one attention budget

What it is. Rollbar emits three fundamentally different signals that most teams flatten into one Slack channel: a new item (a stack trace never seen before), a reactivated item (a resolved item that came back), and an occurrence spike (a known item suddenly firing at 50x its baseline). They do not deserve equal attention.

• A new item after a deploy is the single highest-value signal Rollbar produces — it's a regression announcing itself, usually within minutes of the code that caused it.
• A reactivated item means either the fix didn't work or a rollback/partial deploy resurrected old code. Both are human-worthy, immediately.
• An occurrence spike on a long-known item is usually an environmental symptom (a dependency down, a retry storm, a bad actor), not a new bug. It matters, but it's an investigation signal, not a fix signal.

Why it happens. The default instinct is to route "all errors" to one channel with one severity. Within weeks the channel is 95% occurrence noise from known items, and the two signals that almost always mean "a human should look now" — new and reactivated — drown in it.

How to detect it. Check whether your project even distinguishes them: Settings → Notifications → Slack (or your channel) and look at the trigger list. Rollbar supports separate rules for New Item, Reactivated Item, 10^nth Occurrence, and High Occurrence Rate (notifications docs). If you have one rule with trigger "Every occurrence" — or a New Item rule with no environment filter, so staging noise pages production humans — this failure mode is live in your project. The fix (per-trigger, per-environment rules) is the core of part two.

Typical cost. This is classic alert-fatigue math: when 19 of 20 pings are known-item noise, response time to the twentieth — the real one — stretches from minutes to days.

3. Grouping gone wrong: one bug as fifty items, fifty bugs as one

What it is. Rollbar groups occurrences into items by fingerprinting the stack trace (grouping docs). When fingerprinting misfires, triage breaks in one of two directions. Over-splitting: one underlying bug shows up as fifty near-identical items — common when a dynamic value (an ID, a memory address, a generated class name) leaks into the message or trace, so every occurrence looks unique. You resolve one item and the "same" error stays active under forty-nine other counters. Over-grouping: fifty genuinely different failures share one generic fingerprint — the classic case is a top-level Error or a framework's catch-all handler — so one item quietly accumulates six digits of occurrences spanning a dozen root causes, and resolving it is meaningless.

Why it happens. Default fingerprinting is a heuristic tuned for typical stack traces. Minified JavaScript without source maps, wrapped exceptions, and log-style messages with interpolated values all defeat it. Nobody notices because grouping failures don't alert — they just corrode the item list's meaning.

How to detect it. Over-splitting: on the Items page, search for a distinctive phrase from a known error's title and count how many separate active items match. More than a handful for one code path means split fingerprints. Over-grouping: sort Items by Occurrences and open the top item's Occurrences tab — if the individual traces show different exception classes or different innermost frames, that item is a bucket, not a bug. You can also hunt for catch-all classes in RQL:

SELECT body.trace.exception.class, count(*) AS occurrences
FROM item_occurrence
WHERE timestamp > unix_timestamp() - 604800
GROUP BY body.trace.exception.class
ORDER BY occurrences DESC

A generic class (Error, RuntimeException) dominating that list is your cue to add custom grouping rules or merge the split items in the UI.

Typical cost. Wildly variable, but over-grouping is the dangerous direction: a new bug landing inside a muted catch-all item is invisible until customers report it.

4. Deploy tracking: the correlation signal almost nobody wires up

What it is. Most production errors are introduced by deploys — and Rollbar can tell you which deploy, automatically, if you report deploys to it. With deploy tracking enabled, every item shows the last deploy before its first occurrence, the Versions view shows which code version introduced or resolved which items, and "did the release break something?" becomes a lookup instead of a forensic exercise. It is, in our experience, the single most underused feature in the product.

Why it happens. Deploy tracking requires one extra step in the CI pipeline — a POST to the deploy API with the revision SHA — and since error reporting works fine without it, the step never gets prioritized. The team then pays for the omission in every investigation: manually cross-referencing item timestamps against the deploy log, forever.

How to detect it. Thirty seconds: open your project's Deploys page in the left sidebar. If it's empty, or the last recorded deploy is from an engineer's laptop eight months ago, you're running blind. Check the Versions view too — populated version data with code_version set on occurrences gets you regression detection nearly for free.

Typical cost. Deploy-correlated investigation typically turns a 30–60 minute "when did this start and what shipped around then" hunt into a two-minute confirmation. Multiply by every incident this quarter.

5. Error budgets vs. mute-everything culture

What it is. Faced with the noise from failure modes 1–4, teams reach for the Mute button. Muting is legitimate for truly unactionable items — third-party script errors, bot traffic — but muting spreads. Once muting an item is the normal response to annoyance, the muted set grows monotonically, and eventually the project's real error rate lives almost entirely inside it. The opposite discipline is an error-budget mindset: decide how much error volume per service is acceptable, watch the trend, and treat sustained budget burn as work to schedule — not sound to silence.

Why it happens. Mute is instant relief with an invisible cost. There's no periodic "review muted items" ritual in most teams, so mutes are forever, and a muted item that later 10x's in volume stays silent.

How to detect it. On the Items page, set the status filter to Muted and read the count next to your Active count. A muted set that rivals or exceeds active — or contains high-occurrence items nobody can justify — means muting has become policy rather than exception. Skim the top ten muted items by occurrences and ask one question of each: would we want to know if this doubled? For most, the honest answer is yes.

Typical cost. This is where the outage-in-the-inbox story from the top usually lives: the predictive item wasn't unseen, it was muted six months earlier by someone who left.

Why triage-by-heroics doesn't scale

None of the five problems above is exotic, and each detection step takes minutes. The pattern behind all of them is the same: Rollbar faithfully captures signals, and converting signals into decisions takes sustained human attention that no one has explicitly budgeted. So triage happens in bursts — after an incident, before an audit — and decays back to graveyard state within a quarter.

The durable fix is structural: route the three signal types differently, fix the fingerprints that lie, wire deploy tracking into CI, and put a floor under the mute button. Rollbar's native tooling can do a surprising amount of this — notification rules, the deploy API, RQL, auto-resolve on deploy — and part two of this series builds that DIY setup step by step, with exact API calls.

Make the triage continuous

Everything you just detected manually can be watched continuously. CloudThinker connects to Rollbar read-only and its agents monitor new items, reactivations, and occurrence spikes as they happen — reading the stack trace, correlating with the deploy that introduced it, and proposing a diagnosis with evidence, while every action beyond investigation waits for your approval. Try CloudThinker free — 100 premium credits, no card required — or continue with the native-tools automation guide.