pub fn least_loaded_select(candidates: &[InferenceServerInfo]) -> Option<usize>Expand description
Select the least-loaded candidate (lowest active_sessions, ties broken by
most free GPU memory). Returns None for an empty set.
pub fn least_loaded_select(candidates: &[InferenceServerInfo]) -> Option<usize>Select the least-loaded candidate (lowest active_sessions, ties broken by
most free GPU memory). Returns None for an empty set.