Expand description
Generation quality metrics for self-supervised training
This module provides O(1) online metrics accumulation during text generation, capturing quality signals from logits that can be used for reinforcement learning.
Key metrics:
- Perplexity: Model confidence (lower = more confident)
- Entropy: Decision uncertainty (lower = more certain)
- Entropy variance: Consistency of confidence (lower = more consistent)
- Repetition ratio: N-gram repetition detection (lower = less repetitive)
Structs§
- Generation
Metrics Accumulator - O(1) online metrics accumulator using Welford’s algorithm
- Generation
Quality Metrics - Aggregated quality metrics from a generation (~24 bytes)
- Session
Metrics - Per-session metrics storage for tracking quality across conversation turns