Our vLLM deployment had an HPA targeting 70 percent CPU. Under a load spike, requests queued for 40 seconds while CPU sat at 15 percent, so nothing scaled. GPU-bound servers need to scale on queue depth. We switched to KEDA on vLLM's waiting-requests metric and fixed warm-up.
Requests decide where a pod is scheduled and how much CPU it is guaranteed; limits decide when it is throttled or killed. This guide explains QoS classes, CFS throttling and OOM kills, and shows how we size containers from real metrics with HPA and VPA.
During a traffic spike our checkout service stayed at 3 pods while CPU was pegged. kubectl get hpa showed <unknown>/70% for weeks. A sidecar container added by a platform change had no CPU request, so utilisation could not be computed. Requests on every container fixed it.