Last quarter one of our checkout services had p99 latency spikes every few minutes while its CPU graph sat comfortably at 40 percent of its limit. Another service was restarted 30 times a day with OOMKilled even though "it only uses 300 MB". Both problems came from misunderstanding the two numbers every Kubernetes container spec carries: requests and limits.

This guide is for engineers who write deployment manifests but have never had to explain what those numbers do at the kernel level. By the end you should be able to read a pod's QoS class, tell throttling from saturation, pick starting values from metrics and know how autoscalers use them.

What requests and limits actually do

The two fields are enforced by different components at different times.

Requests are a scheduling promise. The scheduler places a pod only on a node whose allocatable capacity minus the sum of existing requests can fit the new pod's requests. Actual usage does not matter for scheduling. On the node, CPU requests become a relative weight (cpu.weight in cgroup v2), so when the CPU is contended, each container gets a share proportional to its request.

Limits are a ceiling enforced by the kernel. A CPU limit becomes a CFS bandwidth quota (cpu.max): in every 100 ms period, the container may use at most limit x 100 ms of CPU time across all its threads. A memory limit becomes memory.max; when the cgroup cannot reclaim enough memory to stay under it, the kernel OOM killer kills a process inside the container.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: checkout-api
spec:
  template:
    spec:
      containers:
        - name: app
          image: registry.example.com/shop/checkout-api:2025.09.2
          resources:
            requests:
              cpu: "500m"      # half a core guaranteed under contention, used for scheduling
              memory: "512Mi"  # used for scheduling and eviction ranking
            limits:
              memory: "768Mi"  # hard ceiling, exceeding it means OOMKilled
              # no CPU limit on purpose, see below

The key asymmetry: CPU is compressible, memory is not. Running out of CPU makes you slow. Running out of memory makes you dead.

QoS classes and why they matter under pressure

Kubernetes derives a Quality of Service class from the resource spec of every container in the pod:

QoS class Rule Behaviour under node memory pressure
Guaranteed Every container has CPU and memory requests equal to limits Evicted last
Burstable At least one request or limit set, but not Guaranteed Evicted after BestEffort, ranked by usage above request
BestEffort No requests or limits at all Evicted first

You can read it directly:

kubectl get pod checkout-api-7d9f8c6b5-x2k4p -o jsonpath='{.status.qosClass}'

The class also sets the kernel OOM score adjustment, so when the whole node runs out of memory, BestEffort processes are killed first. For a database or anything stateful, Guaranteed is a reasonable default. For stateless web services, Burstable with a sensible memory request is usually the better trade.

CPU throttling: the latency spikes nobody sees on the dashboard

Our checkout service had a 1-core CPU limit, and the CPU usage graph averaged 0.4 cores. But averages over 30 or 60 seconds hide what happens inside each 100 ms window. The service runs on a JVM with a dozen threads. When a burst of requests arrives, eight threads might each run for 20 ms in the same window, using 160 ms of quota where only 100 ms is allowed. The kernel then freezes the whole container for the rest of that period. A request that needed 20 ms of CPU now waits 80 ms more.

The evidence is in the cgroup counters, which cAdvisor exposes:

# Fraction of CFS periods in which the container was throttled
sum by (pod) (rate(container_cpu_cfs_throttled_periods_total{container="app"}[5m]))
/
sum by (pod) (rate(container_cpu_cfs_periods_total{container="app"}[5m]))

Anything above a few percent on a latency-sensitive service is worth acting on. Ours was at 38 percent. From inside a pod you can also read the raw counters:

kubectl exec checkout-api-7d9f8c6b5-x2k4p -- cat /sys/fs/cgroup/cpu.stat
# usage_usec 912334871
# nr_periods 245120
# nr_throttled 93104
# throttled_usec 4120554110

Options, in the order we usually try them:

  1. Remove the CPU limit, keep the request. The container can burst into idle CPU on the node, and the request still guarantees its fair share under contention. This is what we did for checkout; p99 dropped from 840 ms to 190 ms.
  2. Raise the limit well above the request if your platform policy requires limits.
  3. Reduce thread count so the runtime does not try to use more cores than the quota allows. Go since 1.25 sets GOMAXPROCS from the CPU limit automatically, and modern JVMs read the cgroup quota when sizing their thread pools.

Removing CPU limits is not free: a misbehaving pod can use spare CPU that other pods were enjoying. It cannot take CPU that other pods requested, though, because shares are still enforced. For multi-tenant clusters, a namespace LimitRange and ResourceQuota keep the total in check.

Memory: OOMKilled and how to size it

Memory is where limits genuinely protect the node. The second service in our story was a Node.js worker. Its own metric said 300 MB of heap, but the container metric container_memory_working_set_bytes showed 1.1 GB just before each kill, against a 1 GiB limit. The gap was native memory: image processing buffers allocated outside the V8 heap.

kubectl describe pod thumbnailer-5c6d7b8f9-qq7rt | grep -A3 "Last State"
#     Last State:     Terminated
#       Reason:       OOMKilled
#       Exit Code:    137

Rules we follow for memory:

  • Size from container_memory_working_set_bytes, which is what the kubelet uses for eviction, not from the language runtime's heap metric.
  • Set the request close to the steady-state p95 and the limit with headroom for the real peak, typically 1.25 to 1.5 times the observed max.
  • Tell the runtime about the limit. For the JVM, -XX:MaxRAMPercentage=75; for Node.js, --max-old-space-size set below the limit so the heap is collected before the kernel steps in.
  • Keep request and limit equal for memory if you want predictable placement and Guaranteed QoS.

Exit code 137 means the process received SIGKILL. If Reason is OOMKilled, it was the container's own limit. If the pod was evicted instead, the events will say Evicted with a message about node memory pressure, which points at node-level overcommit rather than a single container.

Sizing from metrics, step by step

We size new services with a short, boring process:

  1. Deploy with generous requests and a generous memory limit, no CPU limit.
  2. Run a load test that reaches expected peak plus 30 percent, and let it soak for an hour.
  3. Read p95 and max CPU usage and working set over that window:
quantile_over_time(0.95, sum by (pod) (rate(container_cpu_usage_seconds_total{container="app"}[1m]))[1h:1m])
max_over_time(container_memory_working_set_bytes{container="app"}[1h])
  1. Set the CPU request near p95, the memory request near steady-state working set and the memory limit at max plus headroom.
  2. Revisit after two weeks of real traffic.

For a quick view without Prometheus, kubectl top pod --containers shows current usage from metrics-server, which is enough for a sanity check but not for sizing.

How HPA and VPA fit in

The Horizontal Pod Autoscaler with a CPU Utilization target computes utilization as usage divided by request. If the request is too low, utilization looks high and HPA scales out constantly; if it is too high, HPA never scales.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: checkout-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  minReplicas: 4
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65

The Vertical Pod Autoscaler recommends or applies request values based on observed usage. Running VPA in Auto mode and HPA on CPU for the same deployment makes them fight: VPA raises requests, HPA sees utilization drop and scales in, and so on. We run VPA with updateMode: "Off" purely as a recommender and copy its numbers into manifests during reviews, or we let HPA scale on a custom metric such as requests per second while VPA manages resources.

kubectl get vpa checkout-api -o jsonpath='{.status.recommendation.containerRecommendations[0].target}'

In-place pod resize, which reached beta and is enabled by default in Kubernetes 1.33, allows changing a running container's CPU and memory without recreating the pod. It makes vertical scaling far less disruptive, but treat it as an operational tool rather than a substitute for sensible defaults.

Common mistakes

Mistake Why it hurts Do this instead
No requests at all BestEffort pods get evicted first and the scheduler overpacks nodes Always set CPU and memory requests
Tight CPU limit on a multi-threaded service CFS throttling causes latency spikes at low average usage Drop the CPU limit or raise it well above the request
Sizing memory from heap metrics Native memory and page cache are invisible to the runtime Use container_memory_working_set_bytes
Memory limit far above request on every pod Node overcommit leads to evictions under pressure Keep memory limit close to request
VPA Auto plus HPA on CPU Autoscalers fight over the same signal VPA as recommender, or HPA on a custom metric
Copying values between services Every workload has a different profile Load test and measure per service

Key takeaways

  • Requests drive scheduling and fair share; limits drive throttling and OOM kills.
  • CPU is compressible and memory is not, so treat their limits differently.
  • Watch throttled periods, not average CPU, when latency is spiky.
  • Size memory from working set and leave headroom for the true peak.
  • HPA utilization is relative to requests, so bad requests mean bad autoscaling.
  • Use VPA recommendations to correct your guesses over time.

References