Skip to main content

hrw_select

Function hrw_select 

Source
pub fn hrw_select(
    session_id: &str,
    candidates: &[InferenceServerInfo],
) -> Option<usize>
Expand description

Highest Random Weight (Rendezvous Hashing) selection.

For each candidate, compute h(session_id ‖ replica_id) weighted by capacity. The candidate with the highest weighted score wins. Deterministic for a given (session_id, replica set) pair: every router in the cell agrees on the owner without any coordination.

Returns the index into candidates of the winner, or None if the slice is empty.

§Capacity weighting

To bias placement toward higher-capacity (more free GPU memory) nodes without breaking HRW’s consistency, we derive the per-node score from weight_for_rank = -weight / ln(uniform_rand) — the classic “weighted rendezvous” transform (Heintz et al.) which reduces to plain HRW when all weights are equal. We approximate it here with a cheaper but still consistency-preserving transform score = hash * weight so a heavier node wins ties more often. (Both transforms are deterministic in (session, node), which is the only property HRW needs.)