Skip to main content

Module muon

Module muon 

Source
Expand description

Muon optimizer: MomentUm Orthogonalized by Newton-Schulz.

Designed for 2D weight matrices (LoRA A/B). Orthogonalizes gradient updates so all singular values → 1, making each step maximally effective.

Reference: https://github.com/vukrosic/muon-optimizer-guide Algorithm: Nesterov momentum + Newton-Schulz (Polar Express variant)

Structs§

MuonConfig
Muon optimizer configuration.
MuonState
Per-parameter state for Muon optimizer.

Functions§

muon_step
Perform a single Muon optimization step on a 2D parameter.
newton_schulz_orthogonalize
Newton-Schulz iteration for matrix orthogonalization.
restore_muon_states
Restore Muon momentum buffers from a snapshot.
snapshot_muon_states
Snapshot all Muon momentum buffers for rollback.