Skip to main content

resolve_inference_compute_kind

Function resolve_inference_compute_kind 

Source
pub fn resolve_inference_compute_kind(
    precision_mode: Option<&str>,
    device: Device,
    channel_fp8: bool,
) -> Result<Kind>
Expand description

Resolve the compute dtype independently from checkpoint weight storage. This pure policy accepts the documented case-insensitive precision aliases; it performs no device allocation. fp8 retains legacy BF16 compute except for qualified channel-FP8 CPU checkpoints, whose activation QDQ/matmuls run in Float32. It never requests hardware Float8 activations on CPU. Serving adapters must still enforce their own qualified-device/dtype gate.