Skip to main content

hyprstream_calibration/
disclosure.rs

1//! Pre-committed disclosure language.
2//!
3//! The program's differentiator is falsifiability: every published calibration claim
4//! ships with the same disclosed limits, committed here *before* the numbers exist so the
5//! language cannot drift to fit results. Gates and reports (P0.5, P1.4, P2.3, P3.6)
6//! embed [`DISCLOSURE`] verbatim; [`disclosure_with`] fills the two placeholders every
7//! report must concrete-ize.
8
9/// The pre-committed disclosure text, with `{telemetry_since}` and `{subject}`
10/// placeholders for [`disclosure_with`].
11///
12/// Claims are scoped to (a) mechanically-checkable verifiable-outcome families, (b)
13/// teacher-agreement splits, and (c) realized production telemetry from a stated date —
14/// and calibration-to-teacher is explicitly not calibration-to-reality.
15pub const DISCLOSURE: &str = "\
16DISCLOSURE (pre-committed, System One program).
17
18The calibration and agreement numbers in this report for {subject} are scoped as
19follows, and no stronger claim is made:
20
211. MECHANICALLY-CHECKABLE FAMILIES. Absolute calibration anchors (ECE, Brier, SCE/ACE,
22   RPS, conformal coverage) are measured on procedurally generated verifiable-outcome
23   benchmark families whose ground truth is computed, not judged. They characterize
24   behavior on those families and their stated difficulty strata only.
25
262. TEACHER-AGREEMENT SPLITS. Where ground truth is not mechanically checkable, numbers
27   measure agreement with a corrected teacher ensemble. Calibration-to-teacher is NOT
28   calibration-to-reality: a model can match its teachers' probabilities exactly and
29   inherit their shared errors.
30
313. REALIZED TELEMETRY. Production-outcome calibration, where reported, covers realized
32   traffic since {telemetry_since} only, under the version triple (schema, model,
33   calibration) recorded with each batch. Past calibration does not guarantee future
34   calibration under distribution shift; post-hoc calibration is re-fit online and each
35   re-fit mints a new calibration version.
36
374. MARGINAL VS JOINT COVERAGE. Conformal coverage guarantees are marginal per question.
38   Cross-question joint coverage is measured and reported separately; it is lower than
39   the marginal guarantee whenever per-question errors correlate.
40
415. UNSEEN FAMILIES. Zero-shot transfer to never-synthesized question families is
42   measured on the held-out gate families and published with the same protocol. Transfer
43   is a measured result, not an asserted property.";
44
45/// Render the disclosure with the two required substitutions: the subject (model/arm the
46/// numbers describe) and the start date of the realized-telemetry window (ISO 8601, or
47/// `"no production telemetry yet"` for pre-launch reports).
48#[must_use]
49pub fn disclosure_with(subject: &str, telemetry_since: &str) -> String {
50    DISCLOSURE
51        .replace("{subject}", subject)
52        .replace("{telemetry_since}", telemetry_since)
53}