When Merging LoRA Adapters Stops Being Exact
From My Weird Prompts · 23:40
Show by Daniel Rosehill
Merging a LoRA adapter into its base model is exact in algebra, but finite-precision hardware tells a different story. Folding a half-mebibyte adapter into a 32-mebibyte matrix removes a serving branch rather than shrinking the model, and trades away swappability. The conversation unpacks how bf16 rounding, PEFT's hidden fp32 upcasting, and quantization introduce silent drift—as seen in a documented MiniCPM-V bug—and why combining multiple adapters with methods like TIES and DARE crosses from linear math into lossy surgery. Key sources: • Hugging Face PEFT docs, Model merging, retrieved 2026-11-15, docs index v0.21.2 (updated 2026-11-14). — https://huggingface.co/docs/peft/main/en/developer_guides/model_merging • PEFT docs, Troubleshooting, retrieved 2026-11-15, v0.21.2. — https://huggingface.co/docs/peft/main/en/developer_guides/troubleshooting • PEFT LoRA package reference (`merge_and_unload`, `merge_adapter`/`unmerge_adapter`), retrieved 2026-11-15. — https://github.com/huggingface/peft/blob/main/docs/source/package_reference/lora.md Episode 6619 — myweirdprompts.com/n/6619 (permanent short link: myweirdprompts.com/705826) Listen on Android: MWP FM Player — myweirdprompts.com/apk
More episodes of My Weird Prompts
More episodes
- Setting Limits on Long-Distance Grandparent Visits
- Why Consistency Beats a Perfect Studio Setup
- The Five-Step EQ and Compression Chain for Any Voice
- EQ, Compression, and Limiting for Spoken Voice
- How Python Wheels Work Without Running Code
- How Harvest Now, Decrypt Later Actually Works
- How Implausible Conspiracy Theories Fold In Every Denial
- What Makes an Open-Source Dataset Actually Useful
- Why Brunel the Tortoise Refused a Meccano Drawbridge
- How HBO's Silicon Valley Built a Real Fake Startup
PodSearch is a free, independent podcast search engine. Read about PodSearch, the privacy policy or contact us.