Skip to main content

Module tensor_sampling

Module tensor_sampling 

Source
Expand description

Device-agnostic tensor sampling for token generation

This module implements sampling algorithms directly on PyTorch tensors, working efficiently on both CPU and GPU devices. Despite the historical “GPU” naming, this sampler is fully device-agnostic and respects the device parameter provided during construction.

Structs§

TensorSampler
Device-agnostic token sampler operating on tensors