Skip to main content

Module inference_stage

Module inference_stage 

Source
Expand description

Thread-local execution boundary for one contiguous decoder stage.

A stage owns exactly one global layer range. The first stage receives token IDs and creates embeddings, middle stages receive an activation tensor, and the last stage turns its activation into logits. The stage object and its typed output are deliberately !Send; a future transport boundary must encode activations instead of moving this execution object between threads.

Structs§

InferenceStage
A loaded, same-thread decoder stage.
StageContract
The range and endpoint responsibilities of one loaded decoder stage.
StageSequence
The single cache timeline owned by one loaded stage.
StageTensor
A stage result that remains attached to the engine-owning thread.

Enums§

StageInput
Input admitted at a stage boundary.
StageOutput
Result emitted at a stage boundary.