Advanced Disciplines
Capacity, Bin Packing, and the Lies We Tell the Scheduler
The scheduler is not a magician. It places pods based on the numbers you give it. When those numbers are lies, placement becomes a slow-motion incident.
Text
Authored as doctrine; evaluated as systems craft.
Doctrine
Capacity is the art of making scarcity explicit. In Kubernetes, resource requests are the scheduler’s truth. When teams under-request to win placement, they move cost from planning into incidents.
Kubblai doctrine: treat requests as contracts. If you cannot honor the contract, you are stealing from future uptime.
- Requests drive placement; limits shape runtime behavior; both must be owned.
- Fragmentation is a real cost; perfect packing is an illusion.
- Under scarcity, fairness must be policy (priority classes) not persuasion.
Requests vs limits: the real behavior
Requests reserve. Limits cap. Overcommit is a strategy, not a default. If you overcommit CPU heavily, you accept latency variance. If you overcommit memory, you accept OOM kills and eviction storms.
The Order expects you to name which workloads are allowed to suffer under contention.
- CPU overcommit: plan for throttling; instrument p95/p99 latency.
- Memory overcommit: plan for eviction; monitor node allocatable and pressure.
- Use QoS classes intentionally; do not stumble into BestEffort.
Bin packing and fragmentation
Even if total capacity exists, pods can be unschedulable due to fragmentation: large requests, topology constraints, taints, and anti-affinity. Scarcity is often local, not global.
Mature platforms track headroom per failure domain and per node pool, not only totals.
- Track allocatable headroom by node pool/zone.
- Beware anti-affinity defaults that force spread and waste capacity.
- Use topology spread constraints to encode intent explicitly.
Priority and preemption: hard choices, written down
Under scarcity, the scheduler must decide who runs. Priority classes and preemption encode institutional values: what is essential, what is optional, and what can be displaced.
If you do not define priorities, scarcity will define them for you—through outages.
- Define priority classes per workload tier and tenant class.
- Test preemption behavior in staging; observe disruption impacts.
- Protect critical control-plane adjacent workloads (DNS, ingress, telemetry).
Operator practice
Capacity planning is not a spreadsheet. It is a feedback loop: observe real usage, adjust requests, and validate under peak.
The Order expects a cadence: quarterly request audits, incident-driven recalibration, and published capacity posture.
- Audit top offenders: pods that exceed requests by 2–5×.
- Treat unschedulable events as governance signals, not developer pain.
- Instrument scheduling latency and Pending reasons at fleet scale.
Canonical Link
Canonical URL: /library/capacity-bin-packing-and-the-lies-we-tell-the-scheduler
Related Readings
Advanced Disciplines
LibraryThe Scheduler and the Ethics of Placement
Placement is policy made physical. When you schedule, you are allocating failure domains, cost, and contention.
Advanced Disciplines
LibraryThe Scheduler Under Scarcity: Priority, Preemption, and Hard Choices
When capacity is insufficient, the scheduler becomes governance. Priority and preemption encode institutional values: who runs, who waits, and who is displaced.
Advanced Disciplines
LibraryTaints, Tolerations, and the Law of Affinity
Affinity is desire; taints are refusal. Together they define where work may live and where it must never settle.
Advanced Disciplines
LibraryCluster Autoscaling and the Economics of Expansion
Adding nodes is not ‘scale.’ It is a controlled expansion of failure domains, cost, and operational surface area.
Governance & Power
LibraryPlatform Cost Doctrine: Waste, Density, and the Economics of the Cluster
Cost is a signal. When ignored, it reappears as fragility: overloaded nodes, under-provisioned control planes, and rushed change driven by budget panic.