Performance & Scaling Bug Report

Binance 429 then 418 IP ban: polling the order book burned through request weight

Our monitoring service polled depth and klines for 40 symbols every two seconds. During a volatile hour it hit 429 Too much request weight, kept retrying, and earned an HTTP 418 IP ban that also blocked our trading bot on the same IP. Weight-aware limiting and websockets fixed it.

Ryan Pham Ryan Pham @ryan_bugs · February 27, 2025 · 5 min read · 17 Views
Banner: Binance API IP ban after exceeding request weight limits with polling

On a volatile afternoon, our trading bot suddenly could not place or cancel orders. Every request returned:

HTTP 418
{"code":-1003,"msg":"Way too much request weight used; IP banned until 1740652800000. Please use WebSocket Streams for live updates to avoid bans."}

The bot itself was not making many requests. A separate monitoring service, deployed on the same VM, was. Its logs from the previous half hour:

14:02:11 WARN depth BTCUSDT: HTTP 429 {"code":-1003,"msg":"Too much request weight used; current limit is 6000 request weight per 1 MINUTE. Please use WebSocket Streams for live updates to avoid polling the API."}
14:02:11 INFO retrying in 0.5s
14:02:12 WARN klines ETHUSDT: HTTP 429 ...

It retried every 429 after half a second, for 40 symbols.

Reproducing it

The monitoring service polled, every 2 seconds per symbol:

GET /api/v3/depth?symbol=X&limit=1000   weight 50
GET /api/v3/klines?symbol=X&interval=1m weight 2

40 symbols times 52 weight every 2 seconds is 62,400 weight per minute against a limit of 6,000. Normally the service was configured with fewer symbols; someone had added 30 symbols to its config that morning.

What we ruled out

  • The trading bot's own request pattern. Its weight usage was under 300 per minute.
  • A change in exchange limits. The limits in exchangeInfo matched our expectations.

Root cause

Binance limits REST usage by request weight per IP per minute. Heavy endpoints like full-depth order books cost much more than simple ones. Exceeding the limit returns 429; continuing to send requests after 429s leads to automated IP bans with HTTP 418, escalating in length for repeat offenders. Two design mistakes: the monitoring service polled data that is available on websockets, and its retry logic ignored Retry-After and backed off for only half a second. Because both services shared an outbound IP, the trading bot was banned too.

What is happening under the hood

Every REST response includes headers with current usage, for example:

X-MBX-USED-WEIGHT-1M: 4872

When a 429 or 418 is returned, a Retry-After header says how many seconds to wait. Order placement also has separate order count limits. Limits are per IP for weight and per account for orders. The exchange explicitly recommends websocket streams for live market data instead of polling.

Ban durations scale up for repeated violations, which is why ignoring 429s is the worst possible reaction.

Environment

Component Version / value
Services Trading bot and market monitor on the same VM and egress IP
Monitor Python 3.12, httpx, polling every 2 s per symbol
Endpoints /api/v3/depth limit 1000, /api/v3/klines
Symbols 10, then 40 after a config change

Minimal reproduction

The arithmetic is enough: sum endpoint weights times request rate, compare with the per-minute weight limit in exchangeInfo.rateLimits.

The fix

1. Stop polling live data. The monitor now subscribes to websocket streams: @depth20@100ms partial book streams and @kline_1m streams, combined into a few connections. REST is used only for an initial snapshot or recovery.

2. A shared, weight-aware limiter. Any remaining REST calls go through a process-wide limiter that knows endpoint weights and tracks the server-reported usage:

class WeightLimiter:
    def __init__(self, limit_per_min: int, safety: float = 0.7):
        self.budget = int(limit_per_min * safety)
        self.used = 0
        self.blocked_until = 0.0

    def update_from_headers(self, headers):
        if "x-mbx-used-weight-1m" in headers:
            self.used = int(headers["x-mbx-used-weight-1m"])

    async def acquire(self, weight: int):
        while True:
            now = time.time()
            if now < self.blocked_until:
                await asyncio.sleep(self.blocked_until - now)
                continue
            if self.used + weight <= self.budget:
                self.used += weight
                return
            await asyncio.sleep(1)

    def on_limit_response(self, status: int, headers):
        retry_after = int(headers.get("retry-after", "60"))
        self.blocked_until = time.time() + retry_after
        if status == 418:
            alert(f"IP banned for {retry_after}s")

On 429 or 418, all REST calls stop until Retry-After passes. No retries in between.

3. Separate egress for trading. The trading bot now runs on its own host with its own outbound IP, so analytics workloads cannot get it banned. Its API key's IP allowlist was updated accordingly.

4. Config validation. The monitor computes its expected weight per minute at startup from its config and refuses to start if it exceeds half the limit.

Alternatives and trade-offs

  • Smaller depth limits. limit=100 costs much less weight than 1000; a quick improvement if you must poll.
  • Market data from a third-party feed. Removes exchange limits from the equation, with its own cost and latency.

How to test the fix

A unit test simulates responses with rising X-MBX-USED-WEIGHT-1M and a 429 with Retry-After: 30, and asserts that no requests are sent during the wait. In staging, the monitor ran with 40 symbols on websockets for a week with REST weight under 200 per minute.

Confirming in production

  • No 429 or 418 responses since the change.
  • The trading bot's IP has never been affected by monitoring traffic again.
  • Market data latency in the monitor improved from up to 2 seconds (polling interval) to around 100 ms.

Regression guard

  • Metric: X-MBX-USED-WEIGHT-1M per host, alert at 60 percent of the limit.
  • Alert on any 418 immediately.
  • Startup weight budget check in every service that calls the REST API.

Security implications

An IP ban on the trading host is an availability incident for a system that needs to cancel orders. Isolating trading from other workloads protects that path. API keys for trading are restricted to the trading host's IP and cannot withdraw. Treat exchange rate limits as part of your risk controls, not just a performance detail.

Prevention checklist

  • Use websocket streams for live market data.
  • Know endpoint weights and budget them per IP.
  • Honour Retry-After and stop all calls during a 429 or 418.
  • Never share an egress IP between trading and analytics workloads.
  • Validate configured load against limits at startup.

References

Add a label

#python #websocket #rate-limiting #binance-api #trading-bot
Written by
Ryan Pham @ryan_bugs

I build and run the platforms other engineers ship on: CI/CD pipelines, Kubernetes clusters, GPU inference services, LLM features in production, and the cloud bills that come with all of it. I have also spent years writing and operating automated trading systems, where a rounding bug or a missed websocket message is not a log line but a position you did not intend to hold. Each post here starts from a real failure: the alert, the exact error, what we ruled out, the root cause, and how we proved the fix. Same path every time: Investigate, Explain, Fix, Verify. No hype, no price predictions, just engineering.

Advertisement

Comments

0

No comments yet. Be the first to comment!

Sign In to post a comment.