Expand description
Muon optimizer: MomentUm Orthogonalized by Newton-Schulz.
Designed for 2D weight matrices (LoRA A/B). Orthogonalizes gradient updates so all singular values → 1, making each step maximally effective.
Reference: https://github.com/vukrosic/muon-optimizer-guide Algorithm: Nesterov momentum + Newton-Schulz (Polar Express variant)
Structs§
- Muon
Config - Muon optimizer configuration.
- Muon
State - Per-parameter state for Muon optimizer.
Functions§
- muon_
step - Perform a single Muon optimization step on a 2D parameter.
- newton_
schulz_ orthogonalize - Newton-Schulz iteration for matrix orthogonalization.
- restore_
muon_ states - Restore Muon momentum buffers from a snapshot.
- snapshot_
muon_ states - Snapshot all Muon momentum buffers for rollback.