Skip to main content

attention_entropy

Function attention_entropy 

Source
pub fn attention_entropy(attn_weights: &Tensor) -> Tensor
Expand description

Compute Shannon entropy of attention distributions.

Input: [B, H, N, N] attention weights → Output: [B, H, N] per-token entropy.

Ported from tnn-transformer-1/src/attention_analysis.py:compute_attention_entropy.