How To

Coralogix Integration Guide: From Alert to Investigated Answer

Most Coralogix triage time goes to assembling evidence, not thinking. Part one shows the three settings that decide how long it takes (alert type, TCO tier routing, saved queries) and walks a checkout error-ratio alert hypothesis by hypothesis with real DataPrime, Lucene and PromQL queries.

9 min read
Cover Image for Coralogix Integration Guide: From Alert to Investigated Answer

Coralogix Integration Guide: From Alert to Investigated Answer

The alert lands at 14:07. Error ratio on checkout crossed 5% over ten minutes. Before you've written one sentence of diagnosis, you have four tabs open: the Coralogix logs explorer, a metrics dashboard, the trace view, and your CI system's deploy history. The alert told you that something is wrong. The next 25 to 45 minutes are you working out what, and that gap between firing and understanding is where most observability budgets quietly disappear.

A good Coralogix integration shrinks that gap: data routed to the right TCO tier, alerts defined on signals that predict incidents, DataPrime queries written before you need them. A bad one does the opposite. You get alerts nobody trusts, data too cold to query when it matters, and a bill that grows faster than traffic. This guide covers the parts that decide which one you get, with the queries and console paths to check your own account today.

This is part one of a three-part series. Part two builds DIY alert automation with Coralogix's native tools: webhooks, the Alerts API, TCO policies. Part three covers putting an AI agent on the receiving end of every alert.


The triage math nobody writes down

Count last month's pages. Multiply by the time from acknowledgment to "we know the cause." For most mid-market teams that's 20 to 60 minutes per incident, and most of it isn't thinking. It's assembling: querying logs around the window, checking whether a deploy landed, pulling the slow traces, comparing this Tuesday to last Tuesday.

On Coralogix, three settings decide how big that number is:

01Alert type

Cause, not symptom

A ratio or new-value alert points at the cause. A raw threshold only says something is loud.

02Data tier

Hot where it counts

The log line you need at 14:07 is either instant, slow from the archive, or missing.

03Saved queries

Written in advance

A query saved from the last incident takes ten seconds. One written under pressure takes ten minutes.

Three settings decide how long triage takes on Coralogix. This guide covers each.

Why the default setup keeps the gap wide

Most Coralogix accounts weren't designed. They accreted. The structural reasons triage stays slow are the same almost everywhere:

  • Alerts are written against counts. "More than 100 errors in 10 minutes" was right for the traffic on the day someone wrote it. At peak it fires on noise. At 3 a.m. it can't fire at all, because there aren't 100 requests to fail.
  • Alerts fire on symptoms, one at a time. Each alert describes one signal. The narrative (deploy, then errors, then latency) is something the on-call rebuilds by hand from three separate pages.
  • Data routing was never revisited. Either everything sits in the most expensive tier, or someone cut costs once and pushed investigation context into the archive.
  • The investigation lives in people's heads. The five queries that find the cause get retyped from memory at every incident, with the same syntax errors.

None of this is a Coralogix limitation. It's tuning debt, and you can pay most of it down with the platform's own features. Take the three levers in order.


The four alert types that carry real incidents

Coralogix ships several alert types. Four do most of the work in production:

01Threshold

More or fewer

Counts past a line. The fewer-than direction catches jobs that die silently.

02Ratio

Errors over requests

Means the same thing at peak traffic and at 3 a.m.

03New value

Never seen before

Pages on the first unseen error code, namespace or country.

04Flow

Ordered sequence

Deploy, then errors, then latency: one page that already tells the story.

Four alert types carry most real incidents. Most accounts only use the first.

Standard (threshold) alerts fire when log lines matching a query cross a count: more than 100 ERROR lines from payment-service in 10 minutes, or the inverse, fewer than one heartbeat line in 5 minutes. That second direction is how you catch a dead consumer that fails silently, and it's chronically underused. Absence of logs is often the loudest signal you have.

Ratio alerts divide one query by another. The canonical example is errors over total requests per service. A fixed threshold of "100 errors" is noise at peak traffic and blindness at 3 a.m. A ratio of "errors above 2% of requests" means the same thing at both. If you migrate only one alert this quarter, move your top-of-funnel error alert from a count to a ratio.

New-value alerts fire when a field takes a value not seen in the lookback window: a new error_code, a new Kubernetes namespace writing errors, a new country in your auth failures. They're the cheapest early warning Coralogix offers, because they catch novel failure modes no threshold anticipates.

Flow alerts chain other alerts into an ordered sequence within a time window: a deploy event, then an error-ratio breach, then a latency alert, in that order, within 20 minutes. One flow alert can replace three pages with one page that already tells the story. They take effort to design, so save them for your two or three recurring incident shapes.

Route each alert through a notification group keyed on subsystemname or pod, so one bad node produces one notification per offender instead of one undifferentiated storm.


TCO tiers: route data wrong and you pay twice

Coralogix prices ingestion by use case through the TCO Optimizer, which assigns data (per application, subsystem and severity) to one of three tiers:

TierWhat you getWhat it's for
Frequent SearchFully indexed, hot, instant queryData you actively debug with daily
MonitoringAlerts, dashboards and anomaly detection, processed in-stream, not indexed for fast searchData you watch but rarely grep
ComplianceArchived to your object storage, queryable from the archive (slower)Audit trails, data you keep because you must

The price gap between tiers is large: Frequent Search costs a multiple of Monitoring, which costs a multiple of Compliance (current ratios are on Coralogix's pricing page). That gap creates two symmetrical failure modes.

Everything in Frequent Search. The default posture of teams that never opened the TCO screen. You pay full price to index debug logs, load-balancer access lines and health checks nobody has ever searched. Teams doing their first TCO pass typically find 40 to 70% of volume can leave the top tier without anyone noticing.

Overcorrecting into cold tiers. The subtler mistake. Someone pushes INFO logs from a critical service down to Compliance to save money, and at 14:07 the context around your errors sits in the archive with minutes of query latency instead of milliseconds. Alerts still fire on Monitoring-tier data, because Coralogix evaluates in-stream. The investigation that follows the alert needs fast search, and that's exactly what the cheap tiers give up.

The working rule: severity ERROR and above for production services stays in Frequent Search. High-volume INFO and DEBUG from stable services goes to Monitoring. Access logs and audit trails go to Compliance. Revisit it quarterly, because last year's routing reflects last year's architecture.


Walking the 14:07 alert, hypothesis first

When the checkout alert fires, an experienced engineer doesn't start by reading logs. They list what could explain an error ratio jump and what each explanation predicts, then run the one query that tells them apart.

01Scope

Who fails

Errors grouped by subsystem separate a local fault from a shared dependency.

02Mix

What changed

A never-seen error code points at a code change. The same mix, louder, points elsewhere.

03Dependency

One target

Connection errors against one host make it a dependency story.

04Spans

Slow or broken

Slow spans by operation show whether errors are really timeouts.

05Order

What moved first

Latency before errors means saturation. Both after a deploy marker means the deploy.

Each step is one saved query, and each answer decides which reading survives.

Step 1: who is actually failing? Scope the blast radius with DataPrime before anything else. If several subsystems are in the list, a shared dependency is likely. If only checkout is, the problem is local.

source logs
| filter $m.severity == ERROR
| groupby $l.subsystemname aggregate count() as errors
| orderby errors desc
| limit 10

Say it comes back with checkout at 1,840 errors and nothing else above background. That rules out most of the shared-dependency reading and points at the service itself.

Step 2: what changed in the error mix? Group the failing service's errors by code:

source logs
| filter $m.severity == ERROR && $l.applicationname == 'checkout'
| groupby $d.error_code aggregate count() as hits
| orderby hits desc

If a code you've never seen tops the list, you've found your thread, and a new-value alert on error_code would have paged you on it directly. If the mix is the same as yesterday, only louder, suspect load or a dependency rather than a code change.

Step 3: is it one dependency? Lucene still works everywhere DataPrime does, and it's faster to type for a simple filter:

message:"connection refused" AND subsystemName:checkout AND NOT message:healthcheck

A wall of connection refused against one host is a dependency story. Zero hits sends you back to the code-change reading.

Step 4: is it slow, or is it broken? Query spans the same way you query logs. Span duration lives in $m.duration, in microseconds, so 2,000,000 is two seconds:

source spans
| filter $l.serviceName == 'checkout' && $m.duration > 2000000
| groupby $l.operationName aggregate count() as slow_spans, avg($m.duration) as avg_us
| orderby slow_spans desc

Step 5: did the metric move before the logs did? Metric alerts in Coralogix take PromQL, so the same expression works for alerting and for ad-hoc checks:

histogram_quantile(0.95,
  sum(rate(http_request_duration_seconds_bucket{service="checkout"}[5m])) by (le))

If p95 latency climbed ten minutes before the errors started, saturation came first and the errors are timeouts. If latency and errors moved together right after a deploy marker, it's the deploy.

Five questions, five queries, one incident narrative. The teams that resolve fastest aren't better at querying. They stopped writing these from scratch at 14:09.


What's worth alerting on, per signal

Alert volume isn't coverage. A short list that predicts incidents beats a long list that documents them:

  • Logs: error ratio per service (not count); absence of expected lines (heartbeats, job completions); new values on error_code and deploy-adjacent fields.
  • Metrics: p95 and p99 latency from PromQL histograms; saturation (queue depth, connection pool usage), which moves before errors do.
  • Traces: slow-span count against a per-operation baseline; error-flagged spans on the two or three user-facing critical paths only.
  • Cross-signal: one flow alert per recurring incident shape, ordered deploy, then errors, then latency.

Everything else is a dashboard, not a page. Coralogix's Alerts API lets you keep this list in version control instead of in individuals' heads. Part two covers managing definitions as code, plus webhooks that ship query context into Slack with the alert.

Default setup versus deliberate setup

Default Coralogix accountDeliberately configured
Error alertingRaw counts, re-tuned after every traffic changeRatios that mean the same thing at any volume
Novel failuresFound when a threshold eventually tripsNew-value alerts page on the first unseen code
Incident narrativeThree separate pages, stitched by handOne flow alert in deploy, errors, latency order
Data routingEverything hot, or context pushed to the archiveErrors hot, chatter in Monitoring, audit in Compliance
InvestigationQueries retyped from memorySaved queries, run in the first two minutes

The deeper difference is where the knowledge lives. In the default account it's in the heads of the two people who've seen the most incidents. In the deliberate one it's in alert definitions and saved queries anyone can run.

The gap that survives good configuration

Do all of the above and you've built an excellent detection system. The alert arrives richer, the data is in the right tier, the queries are saved. But at 14:07 a person still opens the tabs, runs the five queries, checks the deploy and writes the diagnosis. That's 20 to 45 minutes, multiplied by every page, at whatever hour the pager picks.

Configuration shrinks the triage gap. It doesn't close it. Closing it means putting something on the receiving end of the alert that runs the investigation itself.

Where this series goes next

Part two builds the native automation with exact configs: ratio and flow alerts, outbound webhooks with custom payloads, the Alerts API, and TCO policies that keep alerting data hot. Part three shows CloudThinker connecting to Coralogix read-only and running those five queries (and the ones that come after them) the moment an alert fires, then returning an investigated answer instead of a notification. Every change it proposes starts in Auto Mode's Manual level, waiting for a person to approve it.

Try CloudThinker free with 100 premium credits and no card required, or continue with part two.

coralogixobservabilityloggingalertingdataprimesre