pub struct InferenceDeploymentProfile {
pub isolation: InferenceIsolationProfile,
pub tenancy: InferenceTenancy,
pub compute: InferenceCompute,
pub resources: InferenceResourceBudget,
}Expand description
Explicit inference deployment profile validated before scheduling.
The current in-process path consumes the compute selector. Isolation and resource budgets are the contract for an external launcher; selecting one of those profiles fails closed until that launcher is present.
Fields§
§isolation: InferenceIsolationProfile§tenancy: InferenceTenancy§compute: InferenceCompute§resources: InferenceResourceBudgetImplementations§
Source§impl InferenceDeploymentProfile
impl InferenceDeploymentProfile
Sourcepub fn cpu_demo(
isolation: InferenceIsolationProfile,
tenancy: InferenceTenancy,
) -> Self
pub fn cpu_demo( isolation: InferenceIsolationProfile, tenancy: InferenceTenancy, ) -> Self
CPU-only profile used by the demo.
Sourcepub fn from_env() -> Result<Self>
pub fn from_env() -> Result<Self>
Load the production scheduling profile from explicit environment signals. Unknown values fail startup; they never fall back to weaker isolation or automatic accelerator selection.
Sourcepub fn validate(&self) -> Result<()>
pub fn validate(&self) -> Result<()>
Validate the deployment contract before an engine is loaded.
Sourcepub fn requires_isolated_launcher(&self) -> bool
pub fn requires_isolated_launcher(&self) -> bool
Whether this profile needs an out-of-process launcher.
Trait Implementations§
Source§impl Clone for InferenceDeploymentProfile
impl Clone for InferenceDeploymentProfile
Source§fn clone(&self) -> InferenceDeploymentProfile
fn clone(&self) -> InferenceDeploymentProfile
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Debug for InferenceDeploymentProfile
impl Debug for InferenceDeploymentProfile
Source§impl Default for InferenceDeploymentProfile
impl Default for InferenceDeploymentProfile
Source§impl<'de> Deserialize<'de> for InferenceDeploymentProfile
impl<'de> Deserialize<'de> for InferenceDeploymentProfile
Source§fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>where
__D: Deserializer<'de>,
Deserialize this value from the given Serde deserializer. Read more
Source§impl PartialEq for InferenceDeploymentProfile
impl PartialEq for InferenceDeploymentProfile
Source§fn eq(&self, other: &InferenceDeploymentProfile) -> bool
fn eq(&self, other: &InferenceDeploymentProfile) -> bool
Tests for
self and other values to be equal, and is used by ==.impl Eq for InferenceDeploymentProfile
impl StructuralPartialEq for InferenceDeploymentProfile
Auto Trait Implementations§
impl Freeze for InferenceDeploymentProfile
impl RefUnwindSafe for InferenceDeploymentProfile
impl Send for InferenceDeploymentProfile
impl Sync for InferenceDeploymentProfile
impl Unpin for InferenceDeploymentProfile
impl UnsafeUnpin for InferenceDeploymentProfile
impl UnwindSafe for InferenceDeploymentProfile
Blanket Implementations§
§impl<T> AnySync for T
impl<T> AnySync for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
Compare self to
key and return true if they are equal.§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
Checks if this value is equivalent to the given key. Read more
§impl<T> FutureExt for T
impl<T> FutureExt for T
§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more§impl<T> Pointable for T
impl<T> Pointable for T
§impl<T> PolicyExt for Twhere
T: ?Sized,
impl<T> PolicyExt for Twhere
T: ?Sized,
§impl<T> ServiceExt for T
impl<T> ServiceExt for T
§fn map_response_body<F>(self, f: F) -> MapResponseBody<Self, F>where
Self: Sized,
fn map_response_body<F>(self, f: F) -> MapResponseBody<Self, F>where
Self: Sized,
Apply a transformation to the response body. Read more
§fn trace_for_http(self) -> Trace<Self, SharedClassifier<ServerErrorsAsFailures>>where
Self: Sized,
fn trace_for_http(self) -> Trace<Self, SharedClassifier<ServerErrorsAsFailures>>where
Self: Sized,
High level tracing that classifies responses using HTTP status codes. Read more
§fn trace_for_grpc(self) -> Trace<Self, SharedClassifier<GrpcErrorsAsFailures>>where
Self: Sized,
fn trace_for_grpc(self) -> Trace<Self, SharedClassifier<GrpcErrorsAsFailures>>where
Self: Sized,
High level tracing that classifies responses using gRPC headers. Read more