Expand description
CLI handlers for adaptive ML inference server
Structs§
- Inference
Response - Response structure for LoRA inference
Functions§
- chat_
completion_ via_ api - Perform chat completion via REST API
- create_
http_ client - create_
lora_ via_ api - Create LoRA adapter via REST API
- execute_
sparse_ query - get_
lora_ info_ via_ api - Get LoRA adapter info via REST API
- handle_
commit_ checkpoint - Handle checkpoint commit command
- handle_
config - handle_
embedding_ query - handle_
pretrain - Handle pre-training command - NOT IMPLEMENTED
- handle_
write_ checkpoint - Handle checkpoint write command
- list_
lora_ via_ api - List LoRA adapters via REST API