Tag See all →

#latency

2 stories

Banner: Kubernetes DNS lookups for external hosts slowed by search domain expansion

DevOps & CI/CD Bug Report

Kubernetes external API calls slow and DNS load high: ndots:5 search path expansion

Ryan Pham · Apr 29, 2026 · 6 min read

Calls from our pods to a payment API had 40 to 200 ms of unexplained latency and CoreDNS was the busiest workload in the cluster. Every lookup of api.payments.example went through five search domains first because of ndots:5. FQDNs with a trailing dot and a lower ndots fixed both.

Banner: Triton dynamic batching increasing tail latency while raising throughput

Performance & Scaling Bug Report

Triton dynamic batching doubled throughput and also doubled p99 latency

Ryan Pham · Nov 12, 2025 · 5 min read

Enabling dynamic batching in Triton Inference Server doubled throughput for our ranking model and pushed p99 latency from 38 to 85 ms, breaking our SLO. A large queue delay and preferred batch sizes made requests wait for batches that rarely filled at low traffic. Measured tuning fixed both.