When you have a handful of machines to run workloads on, the default move is to treat them as interchangeable. Give each one the same base image, the same resource limits, the same expectations, and let the scheduler (or your own memory of what’s running where) sort out placement more or less arbitrarily. It’s the path of least cognitive load, and at small scale it usually doesn’t break anything visibly. Everything has enough headroom that mediocre placement just means some slack sitting in the wrong box.
That stops being harmless once a cluster is small enough that headroom itself is the scarce resource, which on three nodes it always is. At that size, node uniformity isn’t a simplification, it’s a decision to ignore information you already have. You know which of your three machines has the fastest storage. You know which one has the most RAM. You know which one gets hammered with constant small writes and which one mostly sits idle except for occasional bursts. Throwing that away and placing workloads round-robin, or by whatever’s most convenient at deploy time, means matching a RAM-hungry stateful service to whichever node happened to be next in rotation, not to the node that has RAM to spare.
The bottleneck is the placement key
The reframe that helps is small: stop asking “which node has room for this” and start asking “what does this workload bottleneck on, and which node is best suited to that specific pressure.” Every workload has a dominant resource axis, even if it also uses the other two. An observability stack that’s constantly ingesting metrics is I/O-bound: it’s doing sustained writes, often small and frequent, and its performance ceiling is disk throughput and latency, not CPU or RAM. A stateful service holding a large working set in memory is RAM-bound: it wants the node with the most memory available, and it doesn’t much care whether that node’s disk is fast or its CPU is modest. A batch job doing heavy computation is CPU-bound and largely indifferent to where its data lives as long as it’s reachable.
Once you name the bottleneck, placement stops being a guess. The I/O-bound workload goes on the node that’s genuinely good at sustained I/O, meaning it either has the fastest storage of the three or the fewest other tenants competing for that storage’s queue depth. The RAM-heavy workload goes wherever memory is least contended, even if that node’s storage is mediocre, because storage speed was never its problem. This sounds almost too obvious stated plainly, and that’s exactly why it’s easy to skip: the instinct to keep every node configured identically feels like the responsible, low-maintenance choice, when in practice it just means every workload gets an average-fit node instead of the best-fit one.
Why identical configuration is the trap
Configuring every node the same way is attractive because it reduces the number of things you have to remember. One base image, one set of resource limits, one mental model — fewer surprises when you SSH into node two versus node three. But that uniformity is a choice to optimize for operator convenience at the cost of workload fit, and on three nodes the cost is not hypothetical. If your I/O-heavy metrics stack lands on the node with the slowest disk purely because that’s where the deploy script pointed this time, you’ve built in a bottleneck that has nothing to do with the workload’s actual resource ceiling and everything to do with an arbitrary placement decision made months earlier.
The asymmetry goes the other way too: over-provisioning a node for a workload that doesn’t need it wastes the one thing a small cluster can’t afford to waste, which is spare capacity elsewhere. A CPU-light, RAM-light utility service doesn’t need to sit on the node with the most memory: that memory is better spent on something that will use it, and the utility service is just as happy on the weaker box.
This isn’t an argument for hand-tuning every workload’s placement forever, and it isn’t a case for exotic scheduling logic on three machines: that would be solving a problem at a scale you don’t have. It’s a smaller, cheaper habit: before deciding where something runs, name what it stresses, and place it where that specific pressure is best absorbed. On a cluster this size, that one habit does more for the effective capacity of your three nodes than any amount of adding CPU or RAM to each of them individually would.
Reading the bottleneck instead of assuming it
Naming a workload’s bottleneck from its description is a reasonable first pass, but the more reliable version of this exercise looks at how a workload behaves once it’s running somewhere, rather than trusting a one-line mental model of what kind of thing it is. “Observability stack” sounds like a clean I/O-bound label, but the real pressure depends on details that don’t show up in that label: how much of the ingestion pipeline buffers in memory before flushing to disk, how aggressively it indexes on write versus deferring that work, whether retention settings mean it’s constantly compacting old data in the background on top of ingesting new data. Two metrics stacks with the same job description can have meaningfully different bottleneck profiles depending on how they’re configured, and the only way to know which one you have is to watch it.
This is why bottleneck-based placement works better as a loop than as a one-time classification. Place the workload on a best-guess node, then watch what that node’s resources do under real load. If the I/O-bound guess was right, disk throughput or queue depth should be the thing closest to its ceiling while CPU and memory sit comfortably below theirs. If instead CPU turns out to be the tight resource (because, say, the ingestion pipeline does more parsing and transformation than expected before anything touches disk), the placement was built on the wrong premise, and no amount of storage speed on that node was ever going to fix it. Catching that requires looking, not just trusting the label a workload’s category suggests.
Where uniform configuration still makes sense
None of this is an argument against uniformity everywhere. Base OS version, patching cadence, monitoring agent configuration, backup tooling: there’s no bottleneck-driven reason to let those diverge between nodes, and letting them drift for no reason just adds operational surface area with no corresponding benefit. The distinction that matters is between configuration that’s genuinely about hardware capability (how much RAM is physically present, how fast the storage really is, how many cores are available) and configuration that’s about operational consistency, like what monitoring stack runs or how logs get shipped. The first category is exactly where matching workload to hardware pays off, because the hardware differences are real and fixed. The second category is where uniformity is still the right default, because there’s no workload-specific reason for one node to be patched differently than another.
Conflating the two is its own trap in the other direction — treating every configuration knob as sacred and workload-specific turns three machines into three bespoke, hand-tuned snowflakes that are individually harder to reason about and collectively harder to keep patched and consistent. The goal isn’t maximum divergence between nodes. It’s divergence exactly where the underlying hardware already diverges, and uniformity everywhere else.
What this doesn’t solve
None of this replaces monitoring, and specifically the metrics that answer “is it slow, and by how much” rather than logs or uptime checks. Naming a workload’s bottleneck up front is an educated guess based on what the workload does (a metrics ingester probably is I/O-bound, a large in-memory index probably is RAM-bound), but “probably” is doing real work in that sentence. The only way to know whether the placement guess was right is to watch the node afterward: is the disk saturated, is memory pressure where you expected, is the CPU-bound job CPU-bound in practice rather than blocked on something you didn’t account for. Placement by bottleneck is a starting hypothesis, not a settled fact, and on three nodes it’s cheap enough to revisit whenever a workload’s shape changes — which, on a system that grows the way homelab-scale infrastructure tends to grow, it eventually will.
The other limit is that this reasoning degrades as workloads stop having one clean bottleneck. A workload that’s simultaneously I/O-heavy and memory-heavy doesn’t have a tidy answer — you’re back to judgment calls about which pressure matters more for that specific service, on that specific node, right now. At three nodes that’s still a tractable, occasional decision. It’s the kind of thing that stops being tractable by hand once the node count and workload count both grow, which is a good marker for when this needs to become a documented policy — or an actual scheduler’s job — instead of something held in one person’s head.
For now, three nodes and a handful of workloads is exactly the scale where doing this by hand, deliberately, beats both extremes: it beats uniform round-robin placement because it uses the information available about each node, and it beats building or adopting a resource-aware scheduler because that would be solving a coordination problem this setup doesn’t have yet. The right amount of process here is “think about the bottleneck before deciding where something runs, write down why,” not a scheduling system with its own configuration surface to maintain. The moment that stops being enough (because there are too many workloads to reason about individually, or because node profiles change often enough that yesterday’s placement decision is stale) is a legitimate signal to invest in something heavier. Until then, adding that machinery early would just be trading one kind of complexity for another, without a workload count that justifies it.