Order updates stopped after an hour: user data stream listenKey expired without keepalive

We deployed a new market-making bot. In the first hour everything looked right: order acknowledgments, fills and balance updates arrived over the user data stream. After that, the bot's view froze. Orders filled on the exchange, but the bot never heard about it and kept quoting as if they were open. The websocket connection itself stayed open; no errors were logged.

Reproducing it

The bot created a listenKey at startup and connected:

listen_key = client.post("/api/v3/userDataStream")["listenKey"]
ws = await connect(f"wss://stream.binance.com:9443/ws/{listen_key}")

and that was all. Fifty-nine minutes after startup, the stream delivered one last message we had not been handling:

{"e": "listenKeyExpired", "E": 1767690061000, "listenKey": "<redacted>"}

After that, the stream carried no more account events.

What we ruled out

• Network disconnects. The connection remained established and pings were answered.
• No activity. Orders were filling on the exchange during the silent period.

Root cause

A user data stream listenKey is valid for 60 minutes unless it is extended with a keepalive request, which the documentation recommends sending about every 30 minutes. Our previous bot had a keepalive task; the new bot was written from scratch and missed it. When the key expired, the exchange stopped delivering account events on it. The websocket stayed open, so connection-level monitoring saw nothing wrong.

What is happening under the hood

The legacy user data stream works in two steps: create a listenKey over REST, then connect to a websocket path containing the key. The key ties the stream to your account and must be kept alive with periodic PUT requests. If it expires, you receive a listenKeyExpired event and must create a new key and reconnect.

Exchanges evolve these mechanisms. Binance announced the retirement of listenKey-based user data streams for spot in favour of subscribing to user data through the WebSocket API with signature-based or token-based authentication, and set a date after which the listenKey endpoints stop working for spot. If you are still on listenKey, plan the migration.

In both models, account event streams are not a durable log: any gap needs reconciliation over REST (see our post on missed fills after reconnects).

Environment

Minimal reproduction

Create a listenKey, connect, never send keepalives, and wait 60 minutes: you receive listenKeyExpired and then silence.

The fix

1. Keepalive and expiry handling (immediate fix).

async def keepalive(client, key_holder):
    while True:
        await asyncio.sleep(30 * 60)
        await client.put("/api/v3/userDataStream", params={"listenKey": key_holder.key})

async def on_message(msg, state, supervisor):
    if msg.get("e") == "listenKeyExpired":
        await supervisor.restart_user_stream()      # new key, reconnect, reconcile
        return
    state.apply_account_event(msg)

Every restart of the user stream triggers reconciliation of open orders, trades and balances over REST before the strategy continues.

2. Freshness monitoring. Connection state is not enough. The bot now tracks the time since the last account event and sends a cheap REST request (open orders) every few minutes to compare. If the exchange shows a change the stream did not report, the user stream is restarted and an alert fires.

3. Migration to the WebSocket API subscription. A month later we moved to subscribing to user data through the WebSocket API with request signing, which removes listenKey management entirely:

await ws_api.send_signed("userDataStream.subscribe.signature", params={})

The session is authenticated with our API key (we use an Ed25519 key), and account events arrive on the same connection. Reconnects still trigger reconciliation.

Alternatives and trade-offs

• Poll REST only for account state. No stream management, slower updates and more request weight.
• Use a client library that manages streams. Fine if it handles expiry and reconnection correctly; we still keep our freshness monitor.

How to test the fix

We ran the bot on the spot testnet for 72 hours with forced disconnects and confirmed continuous account updates. For the listenKey version, a test with keepalives disabled verified that expiry triggers a restart and reconciliation. For the WebSocket API version, a test kills the connection and verifies resubscription and reconciliation.

Confirming in production

• No silent periods in account events since the fix; freshness checks find no unreported changes.
• The migration to the WebSocket API subscription was completed before the listenKey retirement date for spot.

Regression guard

• Alert when no account event has arrived for 15 minutes while orders are open.
• Periodic REST comparison of open orders, with automatic stream restart on mismatch.
• A shared user-stream component used by all bots, so new bots do not rewrite this logic.

What we would do differently

Keep exchange integration code in one shared, maintained component. The keepalive existed in our older bot and was forgotten in the new one because each bot had its own copy of stream handling. Shared components also make migrations, like moving off listenKey, a single change instead of one per bot. We also subscribed the team to the exchange's API changelog and announcement channels, so deprecations reach us before they become incidents.

Security implications

A listenKey grants read access to account events for anyone who has it, which is why it should never be logged or shared. With signed WebSocket API subscriptions, authentication uses the API key, so protect it the same way as for trading (no withdrawal permission, IP allowlist, secret manager). A bot that does not see its fills cannot enforce its risk limits; freshness monitoring is a risk control.

Prevention checklist

• Keep listenKeys alive and handle listenKeyExpired if you still use them.
• Plan migration to the exchange's current user data subscription method.
• Monitor event freshness, not only connection state.
• Reconcile over REST after any stream restart.
• Centralise user-stream handling in one tested component.

References

• Binance Spot user data stream — https://developers.binance.com/docs/binance-spot-api-docs/user-data-stream
• Binance WebSocket API, user data stream requests — https://developers.binance.com/docs/binance-spot-api-docs/websocket-api/user-data-stream-requests
• Binance Spot API changelog — https://developers.binance.com/docs/binance-spot-api-docs/CHANGELOG