#message-queues

Data Engineering Bug Report

One malformed message blocked our order queue for four hours

Ravi Subramanian · Nov 10, 2026 · 7 min read

A single order event with an unexpected null crashed the consumer, went back to the queue, and was delivered again, forever. Behind it, thousands of orders waited. How we found it, why the queue had no redrive policy, and the max receive count, DLQ and redrive process we use now.

Data Engineering Bug Report

One RabbitMQ consumer hoarded 40,000 unacked messages while the others sat idle

Ravi Subramanian · Nov 9, 2026 · 7 min read

We scaled an image processing consumer from one to eight instances and the backlog barely moved. The first consumer had taken almost every message in the queue into its own buffer. Unlimited prefetch, early acks and how basic.qos and manual acknowledgements fixed the distribution.