AI & Machine Learning
Bug Report
LLM maximum context length exceeded: trimming chat history by tokens, not messages
Long chats in our internal assistant failed with a context length error even though we kept only the last 20 messages. One pasted log file was bigger than the whole window. We moved to token-based trimming with a reserved output budget.