Skip to main content

handle_training_infer

Function handle_training_infer 

Source
pub async fn handle_training_infer(
    registry: &RegistryClient,
    policy_transport: TransportConfig,
    model_ref_str: &str,
    prompt: &str,
    image_path: Option<String>,
    max_tokens: Option<usize>,
    temperature: Option<f32>,
    top_p: Option<f32>,
    top_k: Option<usize>,
    repeat_penalty: Option<f32>,
    sync: bool,
    max_context: Option<usize>,
    kv_quant: KVQuantArg,
    signing_key: SigningKey,
    verifying_key: VerifyingKey,
) -> Result<()>
Expand description

Handle training infer command

Runs inference with TTT enabled, making dirty writes to adapter weights.