Performance & Scaling
Bug Report
Semantic cache returned the answer to a different question at 0.92 similarity
We added a semantic cache to our help assistant to cut LLM cost. Within days a user asking how to cancel a subscription got instructions for cancelling an order. The two questions were 0.92 similar. We added exact-match first, scoped keys, intent gating and a measured threshold.