Kubernetes workloads and autoscaling

Requests vs limits: the request is the scheduling reservation

Two numbers, two different jobs, and this module cares about the first. The request is a reservation the scheduler reads: kube-scheduler bin-packs pods onto nodes by summing requests and only places a pod where its request still fits in the node's allocatable capacity. Nothing about the request is a runtime cap. The limit is the runtime ceiling the node enforces (a cpu limit throttles, an over-limit memory container is OOM-killed); kubernetes-operations owns that enforcement side, so lean on it there rather than here.

Because the scheduler counts only requests and never limits, you can deliberately let the sum of limits on a node exceed its capacity. That is overcommit: you pack the node against requests and bet that not every pod bursts to its limit at once. The bet is cheap for CPU, where the worst case is throttling, and risky for memory, where the worst case is an OOM kill, which is why the senior notes treat memory overcommit as the dangerous side of the same trade.

One footgun follows directly: if you set a limit but no request, Kubernetes copies the limit into the request. So limits alone silently sets your scheduling reservation too, and an oversized limit quietly reserves more of the node than you meant.

# latency-sensitive service: reserve CPU for scheduling, do not cap it;
# keep memory bounded because memory over-limit is an OOM kill
resources:
  requests: { cpu: "500m", memory: "256Mi" }
  limits:   { memory: "256Mi" }   # no cpu limit, so bursts use idle node CPU

QoS classes and who gets evicted first

Kubernetes reads your requests and limits and assigns each pod a Quality of Service class. The class is fixed when the pod is created and drives what happens when a node runs low on memory.

• Guaranteed: every container has a cpu request equal to its cpu limit and a memory request equal to its memory limit, all above zero.
• BestEffort: no cpu or memory request or limit anywhere in the pod.
• Burstable: anything in between (at least one request or limit set, but not the full Guaranteed match).

When a node runs out of memory the kubelet reclaims by evicting pods in QoS order.

Wrong "Guaranteed QoS makes the pod faster or gives it priority CPU." QoS is an eviction and out-of-memory ranking, not a speed tier. Guaranteed buys you being evicted last and a better OOM score; it does nothing for throughput, and a Guaranteed pod is still throttled the moment it hits its cpu limit.

What a CPU-based HPA is actually measuring

The HorizontalPodAutoscaler changes the replica count to chase a target. For a CPU target it computes, roughly, desiredReplicas = ceil(currentReplicas x currentUtilization / targetUtilization). The catch is what "utilization" means: it is a percentage of the cpu REQUEST on your containers, averaged across the ready pods. The request is the denominator.

Wrong "A CPU HPA needs a cpu limit to work." It needs a cpu REQUEST, because utilization is measured against the request. If any targeted container has no cpu request, per-pod utilization is undefined and the HPA does nothing for that metric. A common surprise: the HPA sits idle and reports the CPU metric as unknown, and the cause is one container that forgot its request.

The other consequence of "utilization is relative to the request" is that the request size steers the HPA. An oversized cpu request makes every pod look underused, so the HPA scales late. The senior notes cover why even a correctly sized CPU HPA is the wrong signal for spiky traffic.

Requests are a reservation, limits are a leash

The scheduler only ever looks at requests. It sums the requests of a node's pods and places a new pod where its request still fits in allocatable capacity (node capacity minus kubelet and system reservations). Limits are invisible to scheduling, which is the whole basis of overcommit: you can pack a node so the sum of limits far exceeds capacity, betting that not every pod bursts at once. That bet is safe for CPU (a compressible resource: the worst case is throttling) and dangerous for memory (incompressible: the worst case is an OOM kill).

Enforcement is per resource. A cpu limit becomes a CFS quota per scheduling period on the container's cgroup, so a container is throttled within the period once it exhausts its slice, regardless of idle CPU elsewhere on the node. The cgroup substrate itself (how the quota and periods work, how throttling shows up in cpu.stat) belongs to the Linux notes; the Kubernetes-level fact is that a cpu limit trades tail latency for a hard ceiling. Because of that, equal cpu request and limit on latency-sensitive work is usually the wrong reflex: set the request so the scheduler reserves the CPU, and leave the cpu limit off (or well above the request) so short bursts use idle capacity. Keep memory request equal to its limit, since there is no throttling equivalent for memory.

Pod-level requests and limits (a budget for the whole pod rather than per container, beta since 1.34) participate in the same QoS and scheduling rules; a pod-level memory request equal to its pod-level limit can qualify the pod for Guaranteed.

Node-pressure eviction, and how it differs from an OOM kill

Three different mechanisms reclaim memory, and interviews probe whether you can keep them apart.

• Kernel OOM kill: reactive, per container, triggered when a container exceeds its memory limit and the kernel sees pressure. It kills a process, not the pod object; the container restarts under its policy.
• Node-pressure eviction: the kubelet proactively terminating pods when a node signal (memory.available, nodefs.available, imagefs.available, pid.available) crosses a threshold. It sets the pod's phase to Failed and terminates it.
• API-initiated eviction: the Eviction API, used by kubectl drain. This one, and only this one, respects PodDisruptionBudgets.

The kubelet ranks pods for node-pressure eviction in a specific order: first pods whose usage exceeds their requests, then by Pod Priority, then by how far usage is over the request. QoS falls out of this: a BestEffort pod is over its (zero) request the instant it uses anything, so it goes first; a Guaranteed pod living inside its requests goes last.

Wrong "A PodDisruptionBudget will stop the kubelet from evicting my pod under memory pressure." Node-pressure eviction is involuntary. At a hard threshold the kubelet uses a 0s grace period and honors neither the PDB nor terminationGracePeriodSeconds; the disruption only counts against the budget after the fact. Soft thresholds give you eviction-max-pod-grace-period, not the pod's own grace period.

Why a CPU-based HPA lies for spiky loads

A CPU-target HPA is a lagging, smoothing controller, and spiky traffic is exactly the shape it handles worst. Walk the pipeline a burst has to survive before a replica lands:

• Utilization is a percentage of the cpu request, averaged across the ready pods and over the metrics window. A burst that lasts seconds is averaged against the quiet remainder of the window and lands well below target.
• The controller ignores any scaling whose ratio is within a tolerance of 1.0 (0.1 by default), so small excursions are a no-op by design.
• The control loop syncs about every 15s, and it reads from a metrics pipeline that has its own scrape interval (commonly around 15s for Metrics Server). Fresh pods are discounted: their CPU is ignored for an initialization window and unready pods are assumed to consume 0% when computing a scale-up, which dampens the response further.
• Even after the HPA decides, new pods must schedule, start, and pass readiness before they absorb load.

Stack those and a seconds-long spike is over before a new replica serves a request; you see the p99 damage during the burst and the scale-up after it. CPU utilization is a fine autoscaling signal for slow-moving load and a poor one for bursts.

Right scale bursty work on a leading signal, not a trailing average. Queue depth, in-flight request count, RPS, or p99 latency exposed through custom.metrics.k8s.io or external.metrics.k8s.io react to the cause rather than the after-effect; for known-spiky patterns, pre-scale on a schedule or hold warm headroom. Note the scale-DOWN stabilization window (default 300s) is asymmetric on purpose: the HPA is allowed to scale up fast and made to scale down slowly, so tune the target and behavior policies rather than expecting CPU to catch a spike.

VPA and the cluster autoscaler, and where they fight

Two other autoscalers show up at survey depth, and the useful knowledge is how they collide.

The Vertical Pod Autoscaler right-sizes a pod's requests and limits instead of its replica count (an add-on with a stable v1 API, autoscaling.k8s.io/v1, needs Metrics Server). Its update mode is the interesting part: Off only writes recommendations, Initial applies them only at pod creation, and Recreate evicts running pods to apply new requests, which is itself a disruption. In-place pod resize went stable in 1.35, but as of 1.36 the VPA does not yet apply its changes in place, so Recreate mode's eviction is still how a running pod's requests get updated. The hard rule: do not point an HPA and a VPA at the same resource metric. Both would act on CPU, the VPA resizing requests while the HPA scales replicas against those very requests, and they oscillate. Run HPA on a custom or external metric if you also want VPA on CPU or memory.

The Cluster Autoscaler adds and removes NODES, and it too keys off requests, not usage. It scales up when pods are Pending because their requests do not fit any node, and scales down a node when the node is underutilized (default under 50% of allocatable) and its pods can move. This is why HPA and Cluster Autoscaler pair: the HPA adds pods, some go Pending, the Cluster Autoscaler adds a node.

Wrong "The cluster autoscaler will add a node when my nodes are running hot." It does not read CPU or memory usage. If your requests are set far below real usage, nodes can be genuinely overloaded while every request still fits, and the autoscaler sees no reason to grow. Requests are the currency of both scheduling and node autoscaling; wrong requests break both quietly.

Disruption and placement: PDB, spread, priority

A PodDisruptionBudget caps how many pods of a workload can be down from VOLUNTARY disruption at once, expressed as minAvailable or maxUnavailable. A drain calling the Eviction API is refused when the next eviction would breach the budget, so a rolling node upgrade waits instead of taking all your replicas at once. It does nothing about involuntary loss (node crash, node-pressure eviction), which only counts against the budget.

Placement uses two related tools that people conflate. Pod anti-affinity with topologyKey is a hard or soft rule about co-location: a required anti-affinity of one-pod-per-node means that if you ask for more replicas than nodes, the surplus stays Pending forever. Topology spread constraints are the softer, more scalable tool: maxSkew bounds how unevenly pods sit across domains, and whenUnsatisfiable chooses between DoNotSchedule (a hard placement rule) and ScheduleAnyway (a preference). Reach for spread constraints when you want even distribution across zones with some tolerance, and for required anti-affinity only when two pods genuinely must never share a domain.

Priority and preemption close the loop. A PriorityClass maps a name to an integer; the scheduler orders the pending queue by priority, and when a high-priority pod cannot fit, preemption evicts lower-priority pods on a node to make room. Preemption considers PDBs only best-effort: the scheduler prefers victims whose PDBs stay intact but preempts anyway if it must. Set preemptionPolicy: Never for a pod that should jump the queue without evicting anyone, and reserve system-cluster-critical and system-node-critical for the components that must always schedule.