Expand description
LoRA adapter merging strategies
Implements multiple strategies for merging accumulated tenant deltas into permanent adapter files:
- Replace: Completely replace the existing adapter weights
- Additive: Weighted sum of existing and new weights
- DO-Merge: Direction-Only Merging (Yang et al. 2025) — merges magnitude and direction independently for better quality preservation.
Enums§
- Merge
Strategy - Merge strategy for combining tenant delta with existing adapter.
Functions§
- additive_
merge - Additive merge: result = (1-weight) * existing + weight * new
- do_
merge - Direction-Only Merge (DO-Merge) from Yang et al. 2025.
- merge_
state_ dicts - Merge two state dicts using the specified strategy.
- replace_
merge - Replace merge: return a copy of the new tensor.