Skip to main content

make_chat_spawner

Function make_chat_spawner 

Source
pub fn make_chat_spawner(
    signing_key: &SigningKey,
    model_ref: &str,
    available_tools: Option<Vec<Value>>,
    gen_config: Arc<RwLock<ChatGenConfig>>,
) -> StreamSpawner
Expand description

Build a StreamSpawner that drives inference via ModelClient.

Spawns a dedicated OS thread with a single-threaded Tokio runtime per user message. Applies the chat template, starts an authenticated inference stream, and sends ChatEvent variants back through the provided channel.

available_tools is closure-captured and passed to apply_chat_template so the model’s chat template can inject the tool list into the prompt.