Skip to main content

Module handlers

Module handlers 

Source
Expand description

CLI handlers for adaptive ML inference server

Structs§

InferenceResponse
Response structure for LoRA inference

Functions§

chat_completion_via_api
Perform chat completion via REST API
create_http_client
create_lora_via_api
Create LoRA adapter via REST API
execute_sparse_query
get_lora_info_via_api
Get LoRA adapter info via REST API
handle_commit_checkpoint
Handle checkpoint commit command
handle_config
handle_embedding_query
handle_pretrain
Handle pre-training command - NOT IMPLEMENTED
handle_write_checkpoint
Handle checkpoint write command
list_lora_via_api
List LoRA adapters via REST API