pub fn hrw_select(
session_id: &str,
candidates: &[InferenceServerInfo],
) -> Option<usize>Expand description
Highest Random Weight (Rendezvous Hashing) selection.
For each candidate, compute h(session_id ‖ replica_id) weighted by capacity.
The candidate with the highest weighted score wins. Deterministic for a
given (session_id, replica set) pair: every router in the cell agrees on
the owner without any coordination.
Returns the index into candidates of the winner, or None if the
slice is empty.
§Capacity weighting
To bias placement toward higher-capacity (more free GPU memory) nodes without
breaking HRW’s consistency, we derive the per-node score from
weight_for_rank = -weight / ln(uniform_rand) — the classic
“weighted rendezvous” transform (Heintz et al.) which reduces to plain HRW
when all weights are equal. We approximate it here with a cheaper but still
consistency-preserving transform score = hash * weight so a heavier node
wins ties more often. (Both transforms are deterministic in (session, node), which is the only property HRW needs.)