Skip to main content

Module generation_metrics

Module generation_metrics 

Source
Expand description

Generation quality metrics for self-supervised training

This module provides O(1) online metrics accumulation during text generation, capturing quality signals from logits that can be used for reinforcement learning.

Key metrics:

  • Perplexity: Model confidence (lower = more confident)
  • Entropy: Decision uncertainty (lower = more certain)
  • Entropy variance: Consistency of confidence (lower = more consistent)
  • Repetition ratio: N-gram repetition detection (lower = less repetitive)

Structs§

GenerationMetricsAccumulator
O(1) online metrics accumulator using Welford’s algorithm
GenerationQualityMetrics
Aggregated quality metrics from a generation (~24 bytes)
SessionMetrics
Per-session metrics storage for tracking quality across conversation turns