Skip to main content

Module training

Module training 

Source
Expand description

Training utilities and checkpoint management

This module provides:

  • ttt: Test-Time Training for research-valid model adaptation
  • tenant_delta: Per-tenant LoRA delta for isolated TTT
  • delta_pool: Per-tenant delta registry with LRU eviction
  • merge: LoRA adapter merging strategies (DO-Merge, additive, replace) — 1-vs-1 save-time path
  • delta_reducer: order-independent N-way param-server reducer (stack + truncated SVD)
  • quality_filter: Heuristic-based output filtering (NOT for training)
  • checkpoint: Checkpoint management with git integration
  • data_loader: Training data loading utilities

Re-exports§

pub use adaptation_state::AdaptationStrategy;
pub use adaptation_state::DeltaAdaptationState;
pub use adaptation_state::EvictedSnapshot;
pub use adaptation_state::GuardStatus;
pub use adaptation_state::ResolveOutcome;
pub use adaptation_state::WritebackInfo;
pub use checkpoint::CheckpointConfig;
pub use checkpoint::CheckpointInfo;
pub use checkpoint::CheckpointManager;
pub use checkpoint::CheckpointRequest;
pub use checkpoint::TrainingMetrics;
pub use checkpoint::WeightFormat;
pub use checkpoint::WeightSnapshot;
pub use data_loader::ChatTemplateDataLoader;
pub use data_loader::TrainingDataset;
pub use data_loader::TrainingSample;
pub use delta_pool::DeltaPool;
pub use quality_filter::FilterReason;
pub use quality_filter::FilterResult;
pub use quality_filter::QualityFilter;
pub use quality_filter::QualityFilterConfig;
pub use tenant_delta::load_state_dict_from_bytes;
pub use tenant_delta::serialize_state_dict_to_bytes;
pub use tenant_delta::SharedTenantDelta;
pub use tenant_delta::TenantDelta;
pub use tenant_delta::TenantDeltaConfig;
pub use merge::merge_state_dicts;
pub use merge::MergeStrategy;
pub use delta_reducer::rank_cap_for_dtype_bytes;
pub use delta_reducer::reduce_state_dicts;
pub use delta_reducer::Contribution;
pub use delta_reducer::ContributionGate;
pub use delta_reducer::ContributorState;
pub use delta_reducer::DeltaReducer;
pub use delta_reducer::NoopValidation;
pub use delta_reducer::Provenance;
pub use delta_reducer::StackSvdReducer;
pub use delta_reducer::ValidationHook;
pub use ttt::GradientGatingConfig;
pub use ttt::RankOracle;
pub use ttt::RankOracleConfig;
pub use ttt::TTTConfig;
pub use ttt::TTTContext;
pub use ttt::TTTOverrides;
pub use ttt::TTTResult;
pub use ttt::TTTVerifier;
pub use ttt::TestTimeTrainer;
pub use pipeline::compute_ntp_loss_split;
pub use pipeline::forward_train_pipeline;
pub use pipeline::stage_ranges;
pub use pipeline::TrainStage;

Modules§

adaptation_state
checkpoint
Async checkpoint management for training
data_loader
Training data loading and processing for LoRA fine-tuning
delta_pool
Per-tenant delta registry with LRU eviction
delta_reducer
N-way delta aggregation (param-server reducer) — replaces DO-Merge for multi-contributor merges.
merge
LoRA adapter merging strategies
muon
Muon optimizer: MomentUm Orthogonalized by Newton-Schulz.
pipeline
Pipeline-split (2b) training orchestration for TTT-on-split (#316).
quality_filter
Quality Filter Module
tenant_delta
Per-tenant LoRA delta for isolated Test-Time Training
ttt
Test-Time Training (TTT) Module