Skip to main content

delta_rank_utilization

Function delta_rank_utilization 

Source
pub fn delta_rank_utilization(lora_a: &Tensor, lora_b: &Tensor) -> f64
Expand description

Compute delta rank utilization: how much of allocated rank is actually used.

Returns value in [0, 1]: 0 = rank-1 (collapsed), 1 = fully utilized.

Uses SVD of B^T @ B which is [rank, rank] — much cheaper than B @ A which is [out, in]. B accumulates the learned update (initialized to zeros), so its singular value distribution captures how much of the allocated rank is actually being used. When B is zero (fresh delta), returns 0.0.

lora_a: [rank, in_features], lora_b: [out_features, rank]

Called periodically during TTT (every N steps), not on every forward pass.