Skip to main content

compute_ntp_loss_split

Function compute_ntp_loss_split 

Source
pub fn compute_ntp_loss_split(
    stages: &[TrainStage<'_>],
    input_ids: &Tensor,
    delta: Option<&TenantDelta>,
) -> Result<Tensor, Error>
Expand description

Compute the next-token-prediction (NTP) cross-entropy loss for a TTT step over a pipeline split, with the autograd graph intact across device boundaries.

Mirrors TestTimeTrainer::compute_ntp_loss_with_delta (the single-device path) but drives the forward across stages. The returned loss tensor’s backward pass materializes gradients on each parameter’s own device.

input_ids is [1, seq] on the first stage’s device; the loss is computed on that same device (the last stage’s logits are produced on the last stage’s device, and the shifted targets are taken from input_ids, moved to the logits’ device).