#golang

Databases Bug Report

Cache stampede when a hot Redis key expired and pushed Postgres to 100% CPU

Daniel Tran · Mar 20, 2025 · 8 min read

Every ten minutes our catalog API spiked to 4-second responses and the database hit 100% CPU. A hot Redis key expired and 600 requests rebuilt it at once. Single-flight, a short Redis lock, stale-while-revalidate and TTL jitter fixed it.

Cloud & Infrastructure Bug Report

502 and 503 spikes on every Kubernetes rollout: readiness probes, preStop and graceful shutdown

Daniel Tran · Feb 20, 2025 · 8 min read

Every deploy of our Go pricing API produced 30 to 60 seconds of 502 and 503 errors. New pods got traffic before they were ready, and old pods exited while the ingress still routed to them. A readiness probe, a preStop sleep and real graceful shutdown made rollouts clean.