Skip to main content

Module gen

Module gen 

Source
Expand description

Per-family item generators. Every generator is a pure function of (seed, stratum) over the pinned BenchRng stream — published generators + seeds make every item reproducible and immune to teacher-pretraining contamination.

Modules§

arith
Arithmetic word problems. Truth is computed in i64; distractors and near-miss claims are the classic slips (±1, off-by-ten, transposed digits, add/subtract confusion).
jsoncheck
JSON well-formedness / extraction. The NearMiss stratum is near-miss JSON: documents broken by exactly one classic syntax error (trailing comma, unquoted key, single quotes, missing closing brace). Validity is decided by serde_json, so truth is mechanical.
syllogism
Set-logic syllogisms — held-out gate family (firewalled from training synthesis; measured by the P1.4 leg-(e) zero-shot transfer gate). Chains of All premises over synthetic category names; validity is decided by the chain structure, so truth is mechanical.
temporal
Temporal reasoning — held-out gate family (firewalled from training synthesis; measured by the P1.4 leg-(e) zero-shot transfer gate). Date arithmetic uses Hinnant’s days-from-civil algorithm implemented in-crate so truth stays mechanical and dependency-free.
unitconv
Unit conversion (km↔mi, kg↔lb, °C↔°F). Truth is computed with pinned conversion constants and rounded to integers so outcomes are exact.

Structs§

BenchConfig
Benchmark generation configuration. The frozen release pins DEFAULT_SEEDS_PER_STRATUM and SEED_BASE; regenerating with the same values must reproduce the committed manifest bit-for-bit.

Constants§

BASE_STRATA
The base strata every family emits (permutation expansion is mechanical and applied afterwards to choice items only).
DEFAULT_SEEDS_PER_STRATUM
Seeds per (family, base stratum, kind) in the frozen vob-1.0 release.
SEED_BASE
Seed base for the frozen vob-1.0 release.

Functions§

generate_all
Generate the full benchmark: all families × base strata × kinds × seeds, plus the cyclic-permutation expansion of every choice item.