Skip to main content

Module inference_profile

Module inference_profile 

Source
Expand description

Deployment and scheduling contract for inference-engine instances.

Engine isolation is deliberately separate from model architecture. A TorchEngine keeps one contract whether an operator places it in the host process, a tenant subprocess, or an isolated task/microVM.

Structs§

InferenceDeploymentProfile
Explicit inference deployment profile validated before scheduling.
InferenceInstanceId
Tenant/model/replica identity used for placement and endpoint derivation.
InferenceResourceBudget
Per-instance resource and lifecycle limits.

Enums§

InferenceCompute
Accelerator class reserved for each engine instance.
InferenceIsolationProfile
Address-space isolation selected for an inference engine.
InferenceTenancy
Tenancy posture of the deployment hosting inference.

Functions§

inference_iroh_key_purpose
Domain-separated carrier-key purpose for one opaque service instance.