Skip to main content

reduce_state_dicts

Function reduce_state_dicts 

Source
pub fn reduce_state_dicts(
    reducer: &dyn DeltaReducer,
    contributors: &[ContributorState],
) -> Result<HashMap<String, Tensor>, Error>
Expand description

Aggregate N contributors’ full state dicts into one reduced state dict.

Each contributor supplies a (state_dict, provenance, norm_ratios) where the state dict uses the extract_state_dict key convention ("layer.module.lora_a" / "layer.module.lora_b"). The reducer runs the stack + truncated-SVD per layer key and emits reduced factors at the fixed rank R. Effective-rank / Muon-momentum metadata are intentionally dropped (fresh optimizer post-aggregation, per the spike).

Keys present in only some contributors are aggregated over the subset that has them — this is still order-independent and tolerant of contributors that trained different module sets.