Binance API -1021: Timestamp for this request is outside of the recvWindow

Our market-making bot for a few spot pairs runs on a small cloud VM. One Monday it started logging failed order placements, a handful per hour at first:

2024-10-07 08:14:22 ERROR place_order BTCUSDT: {"code":-1021,"msg":"Timestamp for this request is outside of the recvWindow."}

By the afternoon, almost every signed request failed, including order cancellations, which is the dangerous part: the bot could not cancel resting orders when it wanted to.

Reproducing it

We compared the VM's clock with the exchange's server time:

import time, requests

local_before = int(time.time() * 1000)
server = requests.get("https://api.binance.com/api/v3/time", timeout=5).json()["serverTime"]
local_after = int(time.time() * 1000)
rtt = local_after - local_before
offset = server - (local_before + rtt // 2)
print(f"rtt={rtt}ms offset={offset}ms")
# rtt=38ms offset=-1874ms

The VM was 1.87 seconds behind. Checking the time service:

$ timedatectl
System clock synchronized: no
              NTP service: inactive

What we ruled out

• Wrong signing or parameter order. Signature errors return a different code (-1022), and unsigned requests worked.
• Network latency spikes. Round trips were steady at 30 to 50 ms.
• Exchange-side incident. The status page and other clients were fine.

Root cause

The VM image we had built a month earlier disabled systemd-timesyncd as part of a hardening script, and nothing else was synchronising time. The clock drifted a little each day. Binance rejects a signed request when the server receives it more than recvWindow milliseconds after the request's timestamp (default 5,000 ms), and also when the timestamp is too far ahead of server time. With our clock almost 2 seconds behind and default recvWindow, requests were valid only while drift plus latency stayed under the window; as drift grew, failures became constant.

What is happening under the hood

Signed endpoints require a timestamp parameter (milliseconds) and accept an optional recvWindow (maximum 60,000). The server checks, roughly:

if timestamp < serverTime + 1000 and serverTime - timestamp <= recvWindow: process
else: reject with -1021

A client clock that runs behind makes requests look old; one that runs ahead by more than about a second makes them look like they come from the future, which fails with a related -1021 message about the timestamp being ahead of server time.

The point of recvWindow is protection against delayed or replayed requests. For trading, a smaller window is safer: an order that arrives 30 seconds late may be at a stale price.

Environment

Minimal reproduction

sudo timedatectl set-ntp false
sudo date -s "$(date -d '-3 seconds')"
python place_test_order.py   # -1021

(Run this only on a test machine with testnet keys.)

The fix

1. Real time synchronisation. We installed chrony and pointed it at the cloud provider's time service, with a public pool as fallback:

# /etc/chrony/chrony.conf
server 169.254.169.123 prefer iburst minpoll 4 maxpoll 4
pool 2.pool.ntp.org iburst
makestep 1.0 3
rtcsync

$ chronyc tracking | grep -E 'System time|Last offset'
System time     : 0.000012904 seconds fast of NTP time
Last offset     : +0.000003117 seconds

The hardening script now keeps a time service enabled and the VM image has a test for it.

2. Monitor offset against the exchange. The bot measures its offset to exchange server time every minute (as in the script above) and exports it as a metric. If the absolute offset exceeds 500 ms, it stops opening new orders and alerts.

3. Keep recvWindow modest. We left recvWindow at 5,000 ms. Raising it to 60,000 would have hidden the problem and allowed late orders to execute.

4. Treat -1021 explicitly. On -1021, the bot does not blindly retry order placement. It re-measures the offset, pauses new orders if the clock is bad, and for cancellations retries once with a fresh timestamp, because failing to cancel is riskier than a duplicate cancel request.

Alternatives and trade-offs

• Use server time as the base (timestamp = local_time + measured_offset). Common in client libraries and fine as a backup, but correct system time is still needed for logs and for everything else on the host.
• A much larger recvWindow. Masks drift and latency problems, and allows stale orders through.

How to test the fix

On a test VM, we stopped chrony, shifted the clock by 2 seconds, and confirmed the offset alarm fired and the bot paused new orders against the testnet. Restarting chrony corrected the clock within a minute (makestep allows an immediate step for large offsets at start).

Confirming in production

• Offset to exchange server time stays within plus or minus 15 ms.
• Zero -1021 errors in the following month.
• The offset metric is on the bot's main dashboard next to order latency.

Regression guard

• Image build test: timedatectl show -p NTPSynchronized must be yes after boot.
• Alerts on offset above 500 ms and on any -1021 response.
• Trading pauses automatically if the clock is unhealthy.

Security implications

Time is part of request authentication here. Accepting a larger recvWindow widens the window in which a captured signed request could be replayed and in which delayed orders execute at prices you no longer intend. Keeping clocks accurate lets the window stay small. API keys used by the bot have trading permission only, no withdrawal permission, and are restricted to the VM's IP address.

Prevention checklist

• Make sure every trading host synchronises time, and test it in the image.
• Measure offset to exchange server time continuously.
• Keep recvWindow small; fix clocks instead of widening it.
• Handle -1021 deliberately, especially for cancellations.
• Pause trading automatically when time is unreliable.

References

• Binance Spot API, timing security (timestamp and recvWindow) — https://developers.binance.com/docs/binance-spot-api-docs/rest-api/endpoint-security-type
• Binance Spot API error codes — https://developers.binance.com/docs/binance-spot-api-docs/errors
• chrony documentation — https://chrony-project.org/documentation.html
• Amazon Time Sync Service — https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/set-time.html