Skip to main content

Module merge

Module merge 

Source
Expand description

LoRA adapter merging strategies

Implements multiple strategies for merging accumulated tenant deltas into permanent adapter files:

  • Replace: Completely replace the existing adapter weights
  • Additive: Weighted sum of existing and new weights
  • DO-Merge: Direction-Only Merging (Yang et al. 2025) — merges magnitude and direction independently for better quality preservation.

Enums§

MergeStrategy
Merge strategy for combining tenant delta with existing adapter.

Functions§

additive_merge
Additive merge: result = (1-weight) * existing + weight * new
do_merge
Direction-Only Merge (DO-Merge) from Yang et al. 2025.
merge_state_dicts
Merge two state dicts using the specified strategy.
replace_merge
Replace merge: return a copy of the new tensor.