Skip to main content

newton_schulz_orthogonalize

Function newton_schulz_orthogonalize 

Source
pub fn newton_schulz_orthogonalize(g: &Tensor, steps: usize) -> Tensor
Expand description

Newton-Schulz iteration for matrix orthogonalization.

Maps G → UV^T (its orthogonal factor) by converging all singular values to 1, without ever computing an expensive SVD.

Uses the Polar Express variant with pre-optimized coefficients for stable bfloat16 computation. Compute dtype is chosen adaptively: bf16/fp16 stay native for GPU throughput, fp8 is upcasted to float32.

§Arguments

  • g - 2D gradient matrix [rows, cols]
  • steps - Number of Newton-Schulz iterations (default: 5)