Our nightly product description job used the Batch API to save cost. One night the batch expired with 30 percent unfinished, and our importer joined results by line position, so descriptions landed on the wrong products. We now join by custom_id and handle every terminal status.
After we turned on streaming for our assistant, tool calls failed with JSON parse errors and sometimes ran with another call's arguments. We were parsing argument fragments per chunk and keying them by arrival order. Accumulating by index fixed it.
Our CI compared LLM summaries against stored snapshots at temperature 0 and failed about one run in five. Temperature 0 is not a determinism guarantee on hosted models. We replaced string snapshots with property checks and a scored eval.
Our document tagger hit HTTP 429 constantly while sending only 40 requests a minute, far below the request limit. The token-per-minute limit was the real ceiling, inflated by an oversized max_tokens and minute-aligned bursts. How we read the headers and built a client-side limiter.
Our nightly embedding backfill died on the same few documents every night. A handful of chunks were far over the embedding model's token limit because the chunker split by characters and some files were minified JSON. Token-aware chunking and batching fixed it.
Our support agent started failing with a 400 on roughly one conversation in twelve. The model had begun returning several tool calls in one turn and our loop only answered the first. Answering every tool_call_id in order fixed it.
Switching our extraction endpoint to strict structured outputs failed with an invalid schema error. In strict mode every property must be listed in required and additionalProperties must be false. Optional fields became nullable fields and the errors stopped.
Long chats in our internal assistant failed with a context length error even though we kept only the last 20 messages. One pasted log file was bigger than the whole window. We moved to token-based trimming with a reserved output budget.