Expand description
Device-agnostic tensor sampling for token generation
This module implements sampling algorithms directly on PyTorch tensors, working efficiently on both CPU and GPU devices. Despite the historical “GPU” naming, this sampler is fully device-agnostic and respects the device parameter provided during construction.
Structs§
- Tensor
Sampler - Device-agnostic token sampler operating on tensors