pub struct CellRouter { /* private fields */ }Expand description
Router-side load + health state for a cell’s replica set.
Holds the replica set, the affinity map, and per-node health / down-cache. All updates are router-local in v1 (no gossip).
Implementations§
Source§impl CellRouter
impl CellRouter
Sourcepub fn new(policy: PlacementPolicy) -> Self
pub fn new(policy: PlacementPolicy) -> Self
Create a router with the given placement policy and default timers.
Sourcepub fn with_down_ttl(self, ttl: Duration) -> Self
pub fn with_down_ttl(self, ttl: Duration) -> Self
Override the dial-fail negative-cache TTL.
Sourcepub fn with_stall_deadline(self, deadline: Duration) -> Self
pub fn with_stall_deadline(self, deadline: Duration) -> Self
Override the token-stream stall deadline.
Sourcepub fn with_lease_duration(self, d: Duration) -> Self
pub fn with_lease_duration(self, d: Duration) -> Self
Override the affinity lease duration.
Sourcepub fn policy(&self) -> PlacementPolicy
pub fn policy(&self) -> PlacementPolicy
Current placement policy.
Sourcepub fn affinity(&self) -> &SessionAffinity
pub fn affinity(&self) -> &SessionAffinity
Borrow the affinity map (for tests / status).
Sourcepub fn affinity_mut(&mut self) -> &mut SessionAffinity
pub fn affinity_mut(&mut self) -> &mut SessionAffinity
Borrow the affinity map mutably (e.g. to renew/sweep from outside).
Sourcepub fn place(
&mut self,
session_id: &str,
candidates: &[InferenceServerInfo],
now: Instant,
) -> Option<Placement>
pub fn place( &mut self, session_id: &str, candidates: &[InferenceServerInfo], now: Instant, ) -> Option<Placement>
Place a session on a candidate from the replica set.
Affinity: reuses the live binding if present; otherwise HRW-selects over healthy candidates and records a new binding.Spread: always least-loaded-selects over healthy candidates (no affinity consulted).
Down / stalled nodes are excluded. Returns None only if no healthy
candidate remains (the replica set is empty or entirely failed).
Sourcepub fn report_dial_fail(&mut self, replica_id: ReplicaId, now: Instant) -> bool
pub fn report_dial_fail(&mut self, replica_id: ReplicaId, now: Instant) -> bool
Mark a node down after a synchronous dial-fail. The node is
negative-cached for down_ttl and excluded from
subsequent placements. The next place() for an affected session will
rebind to the next HRW choice.
Returns true if this was a new down-mark (the node was previously
healthy), false if it was already down.
Sourcepub fn report_stall(&mut self, replica_id: ReplicaId, now: Instant)
pub fn report_stall(&mut self, replica_id: ReplicaId, now: Instant)
Mark a node stalled (token-stream watchdog fired). Stalled nodes are
excluded from placement until explicitly cleared
(clear_stall). A stall does not start the
down-clock — it is a stronger signal (the node answered but is not
producing tokens).
Sourcepub fn clear_node(&mut self, replica_id: ReplicaId)
pub fn clear_node(&mut self, replica_id: ReplicaId)
Clear a stall / down mark once the node recovers (e.g. a subsequent
healthCheck succeeds, or tokens resume).
Sourcepub fn stall_deadline(&self) -> Duration
pub fn stall_deadline(&self) -> Duration
Token-stream stall deadline currently in effect.
Sourcepub fn release_session(&mut self, session_id: &str)
pub fn release_session(&mut self, session_id: &str)
Record that a session has completed (release its affinity binding).
Sourcepub fn heartbeat(&mut self, session_id: &str) -> bool
pub fn heartbeat(&mut self, session_id: &str) -> bool
Record a heartbeat (renew the lease) for a session. Returns true if a
binding existed and was renewed.
Sourcepub fn is_excluded(
&self,
replica_id: ReplicaId,
now: Instant,
) -> Option<ExcludeReason>
pub fn is_excluded( &self, replica_id: ReplicaId, now: Instant, ) -> Option<ExcludeReason>
Whether a node is currently excluded (down or stalled).
Trait Implementations§
Source§impl Clone for CellRouter
impl Clone for CellRouter
Source§fn clone(&self) -> CellRouter
fn clone(&self) -> CellRouter
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for CellRouter
impl Debug for CellRouter
Auto Trait Implementations§
impl Freeze for CellRouter
impl RefUnwindSafe for CellRouter
impl Send for CellRouter
impl Sync for CellRouter
impl Unpin for CellRouter
impl UnsafeUnpin for CellRouter
impl UnwindSafe for CellRouter
Blanket Implementations§
§impl<T> AnySync for T
impl<T> AnySync for T
§impl<'a, T, E> AsTaggedExplicit<'a, E> for Twhere
T: 'a,
impl<'a, T, E> AsTaggedExplicit<'a, E> for Twhere
T: 'a,
§impl<'a, T, E> AsTaggedImplicit<'a, E> for Twhere
T: 'a,
impl<'a, T, E> AsTaggedImplicit<'a, E> for Twhere
T: 'a,
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T> FutureExt for T
impl<T> FutureExt for T
§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
Source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
T in a tonic::Request