pub struct TestTimeTrainer {
pub config: TTTConfig,
pub gradient_gating: GradientGatingConfig,
/* private fields */
}Expand description
Test-Time Trainer
Adapts the model to input context BEFORE generation using next-token prediction loss on the input itself. This is the core TTT algorithm.
§Thread Safety
Each adaptation run is isolated - scratch weights are created per-request and do not persist between calls.
§Example
let trainer = TestTimeTrainer::new(TTTConfig::default(), Device::Cuda(0));
// Adapt to input BEFORE generation
let result = trainer.adapt(&engine, &input_tokens).await?;
// Now generate with adapted model
let response = engine.generate(request)?;Fields§
§config: TTTConfig§gradient_gating: GradientGatingConfigImplementations§
Source§impl TestTimeTrainer
impl TestTimeTrainer
Sourcepub fn new(config: TTTConfig, device: Device) -> Self
pub fn new(config: TTTConfig, device: Device) -> Self
Create a new TTT trainer with the given configuration
Sourcepub fn with_gradient_gating(self, gating: GradientGatingConfig) -> Self
pub fn with_gradient_gating(self, gating: GradientGatingConfig) -> Self
Create a new TTT trainer with gradient gating configuration
Sourcepub fn is_enabled(&self) -> bool
pub fn is_enabled(&self) -> bool
Check if TTT is enabled
Sourcepub fn ttt_step(
&self,
loss: &Tensor,
delta: &mut TenantDelta,
learning_rate: Option<f64>,
) -> Result<(f64, bool)>
pub fn ttt_step( &self, loss: &Tensor, delta: &mut TenantDelta, learning_rate: Option<f64>, ) -> Result<(f64, bool)>
Perform a single gradient step on a tenant delta using Muon
Steps: backward → compute grad_norm → clip → muon_step → zero_grad
Sourcepub fn adapt_tenant(
&self,
engine: &TorchEngine,
delta: &mut TenantDelta,
input_tokens: &[u32],
overrides: &TTTOverrides,
) -> Result<(TTTResult, HashMap<String, Tensor>)>
pub fn adapt_tenant( &self, engine: &TorchEngine, delta: &mut TenantDelta, input_tokens: &[u32], overrides: &TTTOverrides, ) -> Result<(TTTResult, HashMap<String, Tensor>)>
Adapt a tenant delta to input context using perplexity-gated TTT
This is the tenant-aware entry point for TTT. It:
- Snapshots delta state (for rollback)
- Computes initial NTP loss → perplexity
- Gates step count based on perplexity
- Runs SGD gradient loop on the tenant delta
- Returns metrics (does NOT auto-commit — client decides)
§Arguments
engine- TorchEngine with loaded modeldelta- Tenant’s LoRA delta (mutable for SGD updates)input_tokens- Tokenized input contextoverrides- Per-request TTT overrides from client
§Returns
(TTTResult, pre_snapshot) where pre_snapshot is the state before adaptation
Sourcepub fn train_step(
&self,
engine: &TorchEngine,
delta: &mut TenantDelta,
input_tokens: &[u32],
gradient_steps: usize,
learning_rate: Option<f64>,
max_adaptation_ms: Option<u64>,
) -> Result<TTTResult>
pub fn train_step( &self, engine: &TorchEngine, delta: &mut TenantDelta, input_tokens: &[u32], gradient_steps: usize, learning_rate: Option<f64>, max_adaptation_ms: Option<u64>, ) -> Result<TTTResult>
Run pure training steps on a tenant delta without generation
Used by the trainStep API endpoint for explicit training.
Steps are clamped to max_gradient_steps and time-budgeted.
Trait Implementations§
impl Send for TestTimeTrainer
impl Sync for TestTimeTrainer
Auto Trait Implementations§
impl Freeze for TestTimeTrainer
impl RefUnwindSafe for TestTimeTrainer
impl Unpin for TestTimeTrainer
impl UnsafeUnpin for TestTimeTrainer
impl UnwindSafe for TestTimeTrainer
Blanket Implementations§
§impl<T> AnySync for T
impl<T> AnySync for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
§impl<T> FutureExt for T
impl<T> FutureExt for T
§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more