Skip to main content

soft_target_kl

Function soft_target_kl 

Source
pub fn soft_target_kl(score: &QuestionScore, target: &Tensor) -> Tensor
Expand description

KL divergence KL(target ‖ p) between a target distribution (the corrected teacher-average histogram in P1.4 distillation) and the scorer’s readout for one question. Distribution targets are the native input of the binned-softmax parameterization (S6b2); the target must be non-negative and sum to 1 over the question’s label arity (2 for noul: [P(false), P(true)]).

Computed in log space (Σ t·(log t − log p)) with the target clamped away from exact zero, so zero-probability target bins contribute nothing instead of NaNs.