Tag See all →

#huggingface

3 stories

Banner: LoRA adapter loaded onto a different base model revision than it was trained on

AI & Machine Learning Bug Report

Fine-tuned LoRA adapter produces garbage in production: loaded on the wrong base model revision

Ryan Pham · Mar 25, 2026 · 6 min read

Our LoRA adapter for ticket summarisation worked in evaluation and produced repetitive nonsense in production. Serving loaded the base model by name without a revision, and the hub repository had been updated with a new tokenizer and weights. Pinning base revisions and a load-time smoke test fixed it.

Banner: batched text generation broken by right padding in a decoder-only language model

AI & Machine Learning Bug Report

Batched LLM generation garbles short prompts: right-padding with a decoder-only model

Ryan Pham · Jul 23, 2025 · 5 min read

Our batch classification job with a small open model produced good output for the longest prompt in each batch and nonsense for the rest. Transformers warned that right-padding was detected for a decoder-only model. Left padding, a proper pad token and the attention mask fixed it.

Banner: Hugging Face tokenizers fork warning after Gunicorn preloads the model

Performance & Scaling Bug Report

huggingface/tokenizers: the current process just got forked warning in Gunicorn

Ryan Pham · Nov 14, 2024 · 6 min read

Our embedding API logged a tokenizers fork warning thousands of times a day and tokenisation got slower after we enabled Gunicorn preload. The tokenizer was used in the master process before workers forked. Loading it after fork fixed both.