Expand description
TTN (Test-Time Naturalization) analysis pipeline for model layer profiling.
Provides per-layer LoRA rank allocation. Uses a tiered approach:
- Tier 1: Embedded profiles for known models (zero-cost const lookup) — ranks from ablation perplexity delta (the gold standard).
- Tier 2: Cached profiles from disk (~1ms file read).
- Tier 3: Computed profiles — layer type detected from weight-key
geometry, every layer assigned a flat unvalidated rank (see
[
UNVALIDATED_RANK_CAP] / [GDN_LORA_RANK]) that the runtime utilization oracle narrows from. There is no per-layer spectral analysis; a data-calibrated allocator is gated on the rank-proxy spike (#842).
Ported from tnn-transformer-1/src/entropy.py and attention_analysis.py.
Structs§
- Layer
Analysis - Layer
Profile - Rank
Utilization Tracker - Rolling-window rank utilization tracker.
- Utilization
Summary - Summary statistics for a key’s utilization history.
Enums§
- Layer
Type - Rank
Signal - Signal for rank adaptation.
Functions§
- attention_
entropy - Compute Shannon entropy of attention distributions.
- attention_
sparsity - Compute top-k attention mass fraction (sparsity measure).
- compute_
layer_ profile - Compute a layer profile from weight-key geometry (Tier 3).
- delta_
rank_ utilization - Compute delta rank utilization: how much of allocated rank is actually used.
- find_
embedded_ profile - Registry lookup for embedded profiles. Returns
Nonefor unknown models. - get_
layer_ profile - Get or compute the layer profile for a model.
- load_
cached_ profile - Load cached profile from model directory.
- qwen3_
5_ 0_ 8b_ profile - Embedded Tier-1 layer profile for the single ablated hybrid (full-attention
- save_
cached_ profile - Save computed profile to model directory (atomic: tmpfile + rename).