Skip to main content

attention_sparsity

Function attention_sparsity 

Source
pub fn attention_sparsity(attn_weights: &Tensor, top_k: i64) -> Tensor
Expand description

Compute top-k attention mass fraction (sparsity measure).

Input: [B, H, N, N] attention weights → Output: [B, H, N] per-token sparsity.

Ported from tnn-transformer-1/src/attention_analysis.py:compute_attention_sparsity.