Reconciliation

Kubernetes is built from controllers: loops that observe current state and compare it to desired state stored in the API. When the two differ, controllers act to reduce the difference. This happens continuously. You don’t “run reconciliation”; you design systems that reconcile.

This is why Kubernetes can restart Pods, recreate ReplicaSets, reschedule workloads, and converge after failures—assuming your desired state is correct and your controllers can make progress.

• Deployments reconcile toward desired replicas via ReplicaSets.
• ReplicaSets reconcile toward a count of Pods with matching labels.
• StatefulSets reconcile identity + storage for stateful workloads.
• Jobs/CronJobs reconcile completion semantics rather than steady-state replicas.
• Custom Controllers (Operators) extend the model to domain-specific desired state.

Reconciliation needs observability

When convergence stalls, you must read events, controller logs, and object status. “It should work” is not an investigation.

Make progress measurable

Well-designed controllers report conditions, last transition times, and reason strings. If you build custom controllers, treat status as a first-class interface.

Design for partial failure

Controllers should be idempotent and resilient. They should tolerate retries, restarts, and conflicts—and they should not assume a single, linear world.

• Mutating live objects manually and forgetting to update the source-of-truth manifest (drift by humans).
• Overusing imperative fixes during incidents without capturing intent afterward.
• Writing controllers that assume they own everything, rather than reconciling carefully with other actors.
• Ignoring status fields and relying on guesswork.