Expand description
Question families, holdout designations, and difficulty strata.
The family taxonomy is the unit of the family-level holdout firewall
(plan v1.6): families designated Designation::Gate are never
synthesized into training data — P1.3’s synthesis run consumes the frozen
manifest and must exclude them, and the P1.4 leg-(e) zero-shot transfer
gate measures the model on exactly these families. Designation is decided
NOW, at benchmark freeze, not after seeing results.
Enums§
- Designation
- Whether a family may be synthesized into training data.
- Family
- A question family: a cluster of procedurally generated, mechanically verifiable question shapes sharing one truth procedure.
- Stratum
- The difficulty/robustness stratum an item belongs to. Reported per stratum by the P0.2 measurement protocol.