Checkmk

Checkmk monitors hosts and services. An agent on each host reports what it finds, the server evaluates it against thresholds, and alerts go out when something is wrong. It descends from Nagios and has diverged substantially.

Service discovery is the feature

The improvement over Nagios-style monitoring is automatic service discovery. The agent enumerates what exists on the host — filesystems, network interfaces, running services, hardware sensors — and Checkmk proposes checks for them. Adding a host means installing an agent and accepting the discovered services.

This sounds like convenience and is really about coverage. Monitoring where each check is configured by hand has gaps, because someone has to remember to add the check for the new filesystem, and eventually nobody does. The filesystem that fills is reliably the one nobody was watching. Discovery closes that by default, and re-discovery catches things added later.

For an environment like a mid-sized institutional cluster, where nobody’s full-time job is monitoring, this is the difference between monitoring that exists and monitoring that reflects reality.

What to get right

Thresholds must be tuned or alerts become noise. Defaults are generic, and a filesystem at 85% may be normal for one host and an emergency on another. The failure mode is universal: too many alerts, alerts get ignored, a real one is missed. Tuning is the work, not the setup.

Distinguish “notify someone now” from “record for later.” Most conditions are the second, and treating them as the first is how alert fatigue starts. Notification rules with escalation and scheduled downtime for planned maintenance are what make this manageable.

Piggyback checks let one host report on things without their own agent — network hardware, or VMs through a hypervisor — which is how coverage extends to devices that cannot run an agent.

Its position

Checkmk is host-and-service monitoring in the classic sense: is this thing up, is this threshold exceeded. That is different from metrics-and-dashboards monitoring in the Prometheus and Grafana style, which is better for application-level time series and trend analysis.

For infrastructure — is the disk full, is the service running, is the RAID degraded — Checkmk is more directly useful and needs far less assembly. It is complementary to Wazuh, which watches for security events rather than availability.

See also: Wazuh, SaltStack, the cluster context, and Linux administration in datacenters.