pub fn muon_step(param: &Tensor, state: &mut MuonState, config: &MuonConfig)Expand description
Perform a single Muon optimization step on a 2D parameter.
Algorithm:
- Nesterov momentum: buf = lerp(buf, grad, 1-momentum); g = lerp(grad, buf, momentum)
- Orthogonalize: g = newton_schulz(g) → UV^T (all SVs = 1)
- Weight decay: p *= (1 - lr * wd)
- Scale for rectangularity: scale = sqrt(max(1, rows/cols))
- Update: p -= lr * scale * g