Skip to main content

handle_load

Function handle_load 

Source
pub async fn handle_load(
    model_ref_str: &str,
    max_context: Option<usize>,
    kv_quant: KVQuantType,
    wait: Option<u64>,
    signing_key: SigningKey,
) -> Result<()>
Expand description

Handle load command - pre-load a model with optional runtime config

wait is None for fire-and-forget, or Some(timeout_secs) to subscribe to notification events and wait for model.loaded/model.failed.