“Monitoring” sounds like one thing: a dashboard, a tool, a checkbox you tick once and move on. It’s a word that hides a category error, though: the three signal types that usually get filed under it answer three genuinely different questions, and conflating them is how you end up staring at a green dashboard while a service is down, or chasing a metrics graph to explain something only a log line will ever tell you.
On a small, personal-scale infrastructure platform this matters more than it might seem to at first, precisely because the temptation to run one tool for everything is strongest when the whole stack is one person’s responsibility. It’s tempting to think “I have a dashboard, I’m covered.” The three signal types don’t collapse into each other just because the operator does.
Three questions, not one
Metrics answer: is it slow, and by how much? A metric is a number sampled over time: request latency, CPU load, queue depth, error rate as a percentage of requests. Its whole value is in the trend and the threshold: this endpoint’s p95 latency has crept from 80ms to 400ms over the last two weeks, or this disk’s write latency spikes every night at the same hour. Metrics are cheap to store at high resolution precisely because they throw away the specific event and keep only the aggregate. That’s also their limit: a metric can tell you that something got slow, and roughly when, but it can’t tell you why a single specific request failed.
Logs answer: what happened? A log line is a discrete, timestamped record of a specific event: this request came in, hit this code path, threw this exception, with this stack trace. Where a metric aggregates away detail to stay cheap, a log’s entire value is the detail it keeps. When a metric graph shows an error-rate spike at 03:14, the metric itself can’t tell you which requests failed or why; that’s a question only the logs from that window can answer. Grafana’s own Loki documentation makes this split explicit as a design choice rather than an implementation detail. Loki indexes only a small set of labels per log stream rather than the full text of every line, deliberately keeping log storage cheap in a way that’s structurally similar to how metrics stay cheap, while still preserving the event content that a metric never had in the first place (Loki overview, Grafana Loki documentation).
Uptime/availability checks answer: is it reachable from outside, right now? This is the one that’s easiest to mistake for redundant with the other two, and it isn’t. An uptime check is explicitly an outside-in check: it doesn’t ask what the internal metrics look like or what the logs say happened; it asks the same question an external user is implicitly asking, which is “can I reach this at all, from where I actually am.” The Prometheus project’s own blackbox exporter is a clean illustration of the pattern: it runs probes against a target over HTTP, TCP, DNS, or ICMP from the vantage point of wherever the exporter itself sits, and reports a single probe_success value (1 or 0) plus latency and status code, deliberately separate from whatever the target’s own internal metrics endpoint reports about itself (prometheus/blackbox_exporter, GitHub; Understanding and using the multi-target exporter pattern, Prometheus documentation). A service can have perfectly healthy internal metrics (low latency, no errors, CPU nowhere near saturated) and still be completely unreachable from outside because of a routing problem, a firewall rule, or a DNS misconfiguration sitting entirely outside the service’s own view of itself. Internal metrics have no way to notice that; only a check running from outside the perimeter does.
Each signal type alerts differently, and that’s not incidental
The three-way split isn’t only about storage and query shape — it changes what a sane alert looks like for each one, and mixing them up produces alerting that’s either too noisy or too quiet in predictable ways.
A metrics-based alert is naturally a threshold or a rate-of-change condition: fire when p95 latency stays above some number for some sustained window, fire when error rate as a percentage crosses a line, fire when disk usage crosses 90%. These alerts tolerate noise well because the underlying signal is a smooth number: a threshold with a “for 5 minutes” clause built in is cheap insurance against a single noisy sample triggering a page for nothing. Getting the pairing between a warning threshold and a critical threshold right is its own design problem, separate from the signal-type split above — see Alert Fatigue by Design for what happens once both thresholds fire for the same underlying condition.
A log-based alert is a different shape entirely: it’s usually pattern matching against event content, alerting when a specific exception class appears, or when authentication failures from a single source cross some count in some window. This is powerful for catching things a metric would never surface (a specific, rare error condition that doesn’t move any aggregate number enough to trip a threshold) but it’s much easier to get wrong in the noisy direction, because log volume and log content vary in ways a smooth time-series doesn’t, and a poorly-scoped pattern match turns into a flood of low-value alerts fast.
An uptime-based alert is the simplest of the three and, not coincidentally, the one that should be treated with the most seriousness when it fires: a failed probe from outside means the thing an actual user would experience is broken, right now, independent of whatever the other two signal types are reporting. This is the alert that shouldn’t be tuned to tolerate noise the way a metrics threshold can be. A single missed external health check might be a blip, but a small number of consecutive failures from an external vantage point is about as close as this stack gets to “an outage is in progress,” and it deserves more urgency than a metrics threshold crossing, precisely because it’s the one signal type measuring the same thing a real user would notice.
Collapsing all three alert types into one alerting pipeline with one severity model tends to blur this. A log-pattern alert and an uptime-check failure can end up with the same visual weight in a shared notification channel, even though one is “something unusual happened, worth a look” and the other is “the service is down right now.” Keeping the three signal types on their own tools doesn’t just keep the data cleaner — it keeps the operator’s sense of “how urgent is this” calibrated to what actually failed.
Why one dashboard for all three is the wrong shape
The reason to keep these as separate tools with separate dashboards, rather than merging everything into one pane of glass, isn’t tooling purism. Each signal type has a different cost profile, a different query language suited to it, and a different failure mode it’s good at catching, and forcing them into one shape tends to make all three worse at once.
A metrics tool (something in the Prometheus/Grafana family, or an equivalent) is built around time-series storage and threshold-based alerting: cheap to sample frequently, efficient to store for months, and queried with something like PromQL that’s built for aggregation and rate-of-change math, not for searching event content. Trying to shove full log text through the same storage model either gets prohibitively expensive at any real retention window, or forces the tool to throw away the text you needed. A log aggregator (a Loki-style or ELK-style stack) is built the opposite way, optimized for ingesting and searching event text and structured fields, generally accepting a higher per-event storage cost in exchange for being able to answer “show me every request from this IP in the last hour” in a way no metrics store is designed to do efficiently. And an external uptime checker is neither of those — it’s a small, dumb, deliberately-external prober whose entire job is to fail loudly the moment it can’t reach you, independent of whatever your own infrastructure believes about its own health.
The failure mode this separation prevents is specific and common: internal metrics and logs both look fine because the service process itself is healthy, while the platform is unreachable from outside for a reason that has nothing to do with the service (a firewall rule change, a DNS record that stopped resolving, a routing change upstream of the host). If uptime checking were treated as just another metric fed into the same dashboard as CPU and latency, it would be easy to let it silently degrade into “another graph that’s usually green” instead of what it needs to be: an independent, outside-in signal that exists specifically because internal signals cannot see certain classes of failure at all.
What this looks like in practice, generically
None of this requires a large team or an elaborate stack. The pattern holds at any scale: a metrics tool scraping a handful of exporters and rendering time-series dashboards, a log aggregator collecting and indexing structured log output from the services that produce it, and a separate, external uptime checker (something as simple as a scheduled prober hitting a health endpoint from outside the network) running independently of both. The important discipline isn’t which specific products fill those three roles; it’s resisting the pull toward a single tool or a single dashboard that quietly starts answering all three questions with only one of the three being checked. When the uptime checker is separate and external by construction, it can’t accidentally become a proxy for internal health, and when the metrics and logs are stored and queried through tools actually built for their respective shapes, neither one has to be squeezed into a format meant for the other.
The category error isn’t fatal on its own; plenty of setups run all three signal types poorly-separated and still catch most problems most of the time. But the gap it opens is specific and predictable: it’s the outage where everything internal looks fine, because “fine” was only ever measured from the inside.
Sources: