Skip to content

Sacred Systems

The API Server as the Gate of Truth

The API is the only public reality in Kubernetes. Everything else is implementation detail and transient effect.

Text

Authored as doctrine; evaluated as operations.

The gate

The API server is the cluster’s boundary of truth: authentication, authorization, admission, and persistence funnel through it.

Kubblai doctrine is blunt: if you do not understand API semantics, you are not operating Kubernetes—you are operating your assumptions.

Admission as a chain of custody

Every write flows through admission. Mutating admissions can change intent; validating admissions can deny it. Both are power. Both must be governed.

Admission failures are control-plane incidents. Treat them as such.

  • Keep webhook latencies low and observable.
  • Have a failure policy and understand its consequences.
  • Version policy changes and rollout safely.

Watch/informer behavior and false confidence

Controllers rely on watches. Watches can disconnect. Resyncs can mask missed events. Your platform’s stability depends on how the control plane behaves under watch pressure.

Serious operators design controllers to tolerate watch churn and to recover deterministically.

QPS, burst, and multi-tenant reality

The API server is finite. Multi-tenant clusters fail when a single tenant can saturate writes or list operations. Rate limits and budgets are governance tools.

If you do not budget API load, you will eventually budget downtime.

Operator practice

Instrument API latency, error rates, inflight requests, and admission time. Correlate with etcd health and controller queue depth.

When the gate is slow, all doctrine becomes difficult.