#pydantic

AI & Machine Learning Bug Report

LLM returns invalid JSON 2% of the time: code fences, truncation and structured outputs

Daniel Tran · Jul 17, 2025 · 9 min read

About 2% of our LLM extraction calls crashed on JSON.parse: markdown code fences, answers cut off at max_tokens and chatty text after the object. We classified every failure, switched to schema-constrained output, checked stop reasons and added validation with one repair attempt.

AI & Machine Learning Bug Report

OpenAI strict structured outputs rejects schema: required must include every key

Ryan Pham · Dec 11, 2024 · 6 min read

Switching our extraction endpoint to strict structured outputs failed with an invalid schema error. In strict mode every property must be listed in required and additionalProperties must be false. Optional fields became nullable fields and the errors stopped.