hyprstream_decision_head/lib.rs
1//! # hyprstream-decision-head — question/criteria encoder + shared option-scorer
2//!
3//! System One program, DAG node **P1.1** (Wave 1). This crate owns the model-side
4//! contract between the pinned jev-1 serialization ([`hyprstream-decision`], P0.1a)
5//! and the converted backbone (P1.2): it turns a runtime-declared
6//! [`QuestionSet`](hyprstream_decision::QuestionSet) into token ids plus an **anchor
7//! map**, and it defines the **shared option-scorer** — the generalist head that reads
8//! one hidden state per anchor position and produces a typed, per-question probability
9//! distribution in a single forward pass.
10//!
11//! ## The generalist contract
12//!
13//! One shared scorer, one forward pass, no cross-question conditioning: every question
14//! is scored **only** from its own anchor positions. Question ids never reach the model
15//! (P0.1a serialization contract), options carry learned special anchor tokens
16//! immediately after their rubric text (S6b1: adjacent-after placement, never a trailing
17//! anchor section), and the scorer applies the *same* linear projection to every anchor
18//! state. Position/option-ID priors are eliminated by construction (learned anchors, not
19//! vocabulary letter tokens — Zheng et al.'s selection-bias mechanism); residual
20//! order-correlation risk is owned data-side by P1.3's mandatory permutation
21//! augmentation, and measured by P0.6's cyclic-permutation stratum.
22//!
23//! ## Readouts per primitive (S6b2-pinned)
24//!
25//! - **noul** — exactly one anchor (P0.1a contract); `P(true) = σ(logit)`, emitted as the
26//! 2-wide `[P(false), P(true)]` distribution matching the Arrow label order.
27//! - **choice** — softmax over the option anchor logits, in canonical authoring order.
28//! - **score** — binned softmax over the per-level anchor logits, trained against
29//! teacher-average histograms (distribution targets; [`scorer::soft_target_kl`] is the
30//! distillation objective for P1.4), with an **expectation readout**
31//! ([`scorer::QuestionScore::expected_value`]) for the probability-weighted value
32//! `Σᵢ i·pᵢ`. The CORN runner-up and its flip conditions are recorded in the plan
33//! (S6b2); this crate implements the primary parameterization only.
34//!
35//! ## Tokenizer-bound budget + learned anchor id (deferred here from P0.1a)
36//!
37//! P0.1a froze the byte-level serialization and the
38//! [`MAX_SERIALIZED_SPEC_TOKENS`](hyprstream_decision::serialize::MAX_SERIALIZED_SPEC_TOKENS)
39//! budget constant; this node binds the budget to a real tokenizer
40//! ([`encoder::DecisionEncoder::encode`] fails with
41//! [`Error::TokenBudgetExceeded`](crate::Error::TokenBudgetExceeded)) and assigns the
42//! learned special anchor token id ([`encoder::assign_anchor_token`]). The anchor id is
43//! recorded per model bundle at conversion time (P1.2); encoding validates that every
44//! anchor placeholder tokenized to exactly the assigned id and that the anchor count
45//! matches the question arities, so a tokenizer/bundle skew fails loudly at encode time
46//! instead of silently misaligning the scorer's readout positions.
47//!
48//! ## Framework choice
49//!
50//! tch-rs (the workspace's `hyprstream/tch-rs` fork) in-workspace, matching the
51//! hyprstream idiom already used by `hyprstream-inference` and the choice P1.2's
52//! conversion consumes: the scorer's `VarStore` checkpoints load directly into the
53//! converted backbone, and the same fork provides the torch-native GDN path pinned for
54//! ROCm/MI210 training (S6c).
55
56pub mod encoder;
57pub mod error;
58pub mod scorer;
59
60pub use encoder::{
61 assign_anchor_token, DecisionEncoder, EncodedRequest, QuestionAnchors, ANCHOR_TOKEN,
62};
63pub use error::Error;
64pub use scorer::{soft_target_kl, QuestionScore, SharedScorer};