pub async fn handle_infer(
model_ref_str: &str,
prompt: &str,
image_path: Option<String>,
max_tokens: Option<usize>,
temperature: Option<f32>,
top_p: Option<f32>,
top_k: Option<usize>,
repeat_penalty: Option<f32>,
seed: Option<u32>,
sync: bool,
signing_key: SigningKey,
) -> Result<()>Expand description
Handle infer command
Runs inference via InferenceService, which:
- Enforces authorization via PolicyManager
- Auto-loads adapters from model directory
- Training is always DISABLED - this is a read-only inference command
For inference with training (TTT), use hyprstream training infer instead.
ยงParameters
signing_key: Ed25519 signing key for request authentication