pub struct LoadedModel {Show 16 fields
pub instance: InferenceInstanceId,
pub model_ref: String,
pub pinned_commit: Option<Oid>,
pub transport: TransportConfig,
pub network_transport: TransportConfig,
pub service_handle: SpawnedService,
pub client: InferenceClient,
pub incarnation: String,
pub work_audience: String,
pub load_generation: u64,
pub router: CellRouter,
pub load_state: Vec<InferenceServerInfo>,
pub loaded_at: Instant,
pub last_used: Instant,
pub ttt_config: Option<TTTConfig>,
pub generation_defaults: SamplingParams,
}Expand description
Information about a loaded model
Fields§
§instance: InferenceInstanceIdTenant/model/replica identity of this inference instance.
model_ref: StringModel reference string (e.g., “qwen3-small:main”)
pinned_commit: Option<Oid>Exact Git commit whose owned base inputs were used, if pinned.
transport: TransportConfigLocal transport for this model’s InferenceService (#320). This remains
the Inproc arm registered by the spawner and is never advertised as a
remotely dialable reach.
network_transport: TransportConfigAdvertised network reach for remote callers. Local dispatch continues
to use transport as the in-process fast path.
service_handle: SpawnedServiceHandle to stop the InferenceService
client: InferenceClientClient for communicating with the InferenceService (built from
transport via dial() — the co-located fast path).
incarnation: StringWorker-generated 256-bit incarnation of THIS running pinned worker, reported through the private readiness handoff for this spawn attempt. Fresh per run: pre-restart work orders fail audience verification at the replacement worker (Sol bounded incarnation plan).
work_audience: StringExact versioned internal-work audience of this incarnation
(iw1/{deterministic service name}/{incarnation}). Minting uses THIS
binding — never a recomputed deterministic name.
load_generation: u64Spawn-attempt generation this binding came from. A delayed result from an older attempt cannot overwrite a newer binding (generation checked at handoff validation).
router: CellRouter#322 leaf cell-router state for this model. Holds the session→owner
affinity map (heartbeat-lease renewal, KV-cache stickiness) and the
per-node load/health counters used by HRW placement. In v1 this is a
single co-located node (the router fast-paths to client); the
load_state set grows to multiple nodes when cross-host replicas are
resolved via the Resolver.
load_state: Vec<InferenceServerInfo>Replica-set snapshot for the router (one entry per known inference server serving this model’s OID). Updated as the Resolver yields new reaches; consumed by HRW placement.
loaded_at: InstantWhen the model was loaded
last_used: InstantWhen the model was last used
ttt_config: Option<TTTConfig>Online training (TTT) configuration (if enabled)
generation_defaults: SamplingParamsGeneration parameter defaults from model’s generation_config.json
Auto Trait Implementations§
impl Freeze for LoadedModel
impl !RefUnwindSafe for LoadedModel
impl Send for LoadedModel
impl Sync for LoadedModel
impl Unpin for LoadedModel
impl UnsafeUnpin for LoadedModel
impl !UnwindSafe for LoadedModel
Blanket Implementations§
§impl<T> AnySync for T
impl<T> AnySync for T
§impl<'a, T, E> AsTaggedExplicit<'a, E> for Twhere
T: 'a,
impl<'a, T, E> AsTaggedExplicit<'a, E> for Twhere
T: 'a,
§impl<'a, T, E> AsTaggedImplicit<'a, E> for Twhere
T: 'a,
impl<'a, T, E> AsTaggedImplicit<'a, E> for Twhere
T: 'a,
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
§impl<T> FutureExt for T
impl<T> FutureExt for T
§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> IntoRequest<T> for T
impl<T> IntoRequest<T> for T
Source§fn into_request(self) -> Request<T>
fn into_request(self) -> Request<T>
T in a tonic::Request