Skip to content

Advanced Disciplines

Traces, Metrics, and the Reading of Omens

Telemetry is a system. If you do not govern cardinality and cost, observability becomes its own outage.

Text

Authored as doctrine; evaluated as operations.

Doctrine

Signals are not mystical. They are measured effects. The ‘omens’ are only patterns you have learned to read under pressure.

Kubblai doctrine: read signals conservatively, corroborate, and act with restraint.

Cardinality is cost

High-cardinality labels and unbounded dimensions can bankrupt your observability platform or crash it at peak traffic.

Govern telemetry like you govern RBAC: with explicit constraints.

Traces for causality

Traces answer a different question than metrics: ‘why is this request slow’ rather than ‘is it slow’. Use traces to connect downstream dependencies and see where time is spent.

Traces without sampling strategy and attribute discipline become expensive noise.

Metrics for saturation truth

Saturation metrics (CPU throttling, queue depth, connection pools, disk IOPS) often predict incidents better than error rates. Build dashboards around saturation and latency budgets.

The strongest engineers can spot systemic saturation early and reduce churn.