Skip to main content

TorchEngine

Struct TorchEngine 

Source
pub struct TorchEngine {
    pub active_lora: Arc<Mutex<Option<String>>>,
    /* private fields */
}
Expand description

PyTorch inference engine using tch-rs

Thread-safe implementation using proper synchronization primitives. All mutable state is protected by mutexes with poisoning recovery.

Fields§

§active_lora: Arc<Mutex<Option<String>>>

Active LoRA adapter name - thread safe with mutex

Implementations§

Source§

impl TorchEngine

Helper functions for tensor operations

Source

pub fn set_seed(&self, seed: u64)

Set the random seed for deterministic generation Useful for debugging - enables reproducible token sequences

Source

pub fn get_vocab_size(&self) -> usize

Get cached vocabulary size without locking

Source

pub fn device(&self) -> Device

Get the device this engine is using for computation

Source

pub fn device_pool(&self) -> Option<&DevicePool>

The resolved multi-device pool, if this engine was built with >1 device.

Returns None for single-device engines (the common case, including the legacy single-gpu_device_id path and CPU). When Some and non-single, the loaded model is built as a single pipeline stage with a layer→device map derived from this pool (#314 wiring). The pool’s primary equals Self::device.

Source

pub fn is_multi_device(&self) -> bool

Whether this engine is configured to build a multi-device pipeline.

True exactly when Self::device_pool is Some and holds >1 device. The model is constructed with LayerDeviceMap::even_split across the pool; single-device engines return false and build the whole model on Self::device.

Source

pub fn initialize_kv_registry( &mut self, num_layers: usize, max_seq_len: usize, quant_type: KVQuantType, memory_budget: Option<usize>, )

Initialize the KV cache registry for session-based cache isolation.

This should be called after loading a model when the configuration is known. The registry will manage separate KV caches for different sessions/requests.

Source

pub fn kv_compat_fingerprint_for( &self, adapter_generation: u64, ) -> Option<KvCompatFingerprint>

Expected KV-cache compatibility fingerprint for the loaded model under the given adapter/delta generation, or None when no authoritative identity is available.

The effective identity folds in the base-weight content digest, the full effective architecture/runtime config, and the live adapter_generation actually applied to the request. Returns None when no descriptor was built (the base-weight digest could not be computed) or when the descriptor is not yet authoritatively complete (tokenizer identity not yet folded in, or tokenizer hashing failed). The session-reuse path treats None as fail-closed: it discards a cache rather than reusing it under a missing or uncertain identity — it never treats absence as “no gating”.

Source

pub fn compute_kv_budget(&self) -> Option<usize>

Compute a KV cache memory budget based on model size and available GPU memory.

Uses HYPRSTREAM_KV_BUDGET_MB env var if set, otherwise estimates: GPU total (from common card sizes) minus model weights, times 0.7. Returns None on CPU (no budget needed).

Source

pub fn kv_registry(&self) -> Option<Arc<KVCacheRegistry>>

Get the KV cache registry (for external access)

Source

pub fn set_session(&self, owner: CacheOwner) -> Result<()>

Set the active session for generation.

This sets which KV cache will be used for the next generation call. Use CacheOwner::Session(session_id) for conversational sessions that should preserve context across multiple requests.

§Example
ⓘ
// For a conversational session (preserves context)
engine.set_session(CacheOwner::Session("conversation-123".into()));

// For a stateless request (cache discarded after)
engine.set_session(CacheOwner::new_stateless());
Source

pub fn get_session(&self) -> Option<CacheOwner>

Get the current active session owner

Source

pub fn clear_session(&self) -> Result<()>

Clear the active session (subsequent generations will use the model’s internal cache)

Source

pub fn release_session(&self, owner: &CacheOwner) -> Result<()>

Release a session’s KV cache (for cleanup after conversation ends)

Source

pub fn session_cache_memory_usage(&self) -> usize

Get total memory usage across all session caches

Source

pub fn active_session_count(&self) -> usize

Get number of active session caches

Source

pub fn evict_session_caches(&self)

Evict LRU session caches to stay within memory budget

Source

pub fn swap_session_cache( &self, tenant_id: Option<String>, weight_epoch: u64, ) -> bool

Swap the active session’s KV cache into the model.

If an active session owner is set and the registry is initialized, retrieves (or creates) the session’s cache from the registry and installs it in the model via set_kv_cache(). Returns true if a session cache was swapped in.

tenant_id is the effective tenant whose delta contributed to this request’s weights (None when no tenant delta contributes), and weight_epoch is that delta’s current weight epoch (ignored when tenant_id is None). The pair is registered against the owner cache before the cache is installed in the model, so the cache is tracked for invalidation (and cleared if its previously-computed KV is stale w.r.t. a different tenant or an older weight epoch) before generation can read a single token from it.

Atomicity invariant (F2): the register→install sequence below is not guarded by a lock spanning both operations. It is race-free today because InferenceService runs on a single-threaded current_thread Tokio runtime inside a LocalSet, and this function is await-free — no other request task can interleave between register_delta_dependency and set_kv_cache, so the pair executes atomically with respect to other requests. If InferenceService ever moves to a multi-threaded runtime (or an .await is introduced here), this invariant breaks and a spanning lock or an epoch handshake would be required.

Source

pub fn model_memory_usage(&self) -> usize

Estimate model memory usage in bytes.

Sums the byte sizes of all parameters in the VarStore.

Source

pub fn new(config: RuntimeConfig) -> Result<Self>

Create new PyTorch engine

Source

pub async fn new_async(config: RuntimeConfig) -> Result<Self>

Create new PyTorch engine (async version)

Source

pub fn get_tensor(&self, name: &str) -> Option<Tensor>

Get tensor from VarStore by name (for inference) - thread safe

Source

pub fn list_tensor_names(&self) -> Vec<String>

List all available tensor names in VarStore - thread safe

Source

pub fn has_varstore(&self) -> bool

Check if model is loaded via VarStore - thread safe

Source

pub fn is_eos_token(&self, token_id: usize) -> bool

Check if a token ID is the EOS token (lock-free using cached ID)

Source

pub fn get_tokenizer(&self) -> Result<Tokenizer>

Get the tokenizer for streaming decoding - CoW makes this cheap!

Returns a cloned tokenizer (cheap due to copy-on-write)

Source

pub fn forward(&self, input_ids: &[i64]) -> Result<Tensor>

Run inference on the model (supports both TorchScript and VarStore models) - thread safe

Source

pub fn forward_cached( &self, input_ids: &[i64], start_pos: usize, ) -> Result<Tensor>

Run optimized inference with KV caching - only process new tokens

Source

pub fn forward_with_delta_cached( &self, input_ids: &[i64], start_pos: usize, delta: Option<&TenantDelta>, ) -> Result<Tensor>

Run inference with KV caching and optional delta injection

Same as forward_cached() but calls forward_with_cache_and_delta() on the model, injecting per-tenant LoRA corrections at each attention layer.

Source

pub fn forward_batched_step( &self, sequences: &mut [(Vec<i64>, usize, Arc<Mutex<KVCacheManager>>)], delta: Option<&TenantDelta>, ) -> Result<Tensor>

Batched ragged decode step over several same-delta sequences (#329).

Each entry is (new_token_ids, start_pos, per-sequence KVCacheManager). Returns last-token logits per row [B, vocab]. Delegates to the model’s forward_batched (Llama-only; other architectures return an error so the caller falls back to per-stream forward_with_delta_cached). Tensors stay on this (bridge) thread; only the Send token/logits metadata crosses.

Source

pub fn forward_with_delta_full( &self, input_ids: &[i64], delta: Option<&TenantDelta>, ) -> Result<Tensor>

Run full forward pass with optional delta injection (no KV cache)

Source

pub fn clear_kv_cache(&self)

Clear KV cache before new generation to prevent context pollution

Source

pub fn snapshot_ssm_states( &self, ) -> Option<(Vec<Option<Tensor>>, Vec<Option<Tensor>>)>

Snapshot Qwen3.5 SSM (conv/rec) states for TTT adaptation.

Returns Some((conv, rec)) if the loaded model is Qwen3.5; None otherwise. Each Tensor is deep-copied via .copy() (C4 fix) so the snapshot is independent of subsequent forward-pass mutations during the TTT loop.

Source

pub fn restore_ssm_states( &self, snapshot: Option<(Vec<Option<Tensor>>, Vec<Option<Tensor>>)>, )

Restore Qwen3.5 SSM states from a snapshot produced by snapshot_ssm_states.

No-op if the loaded model is not Qwen3.5 or snapshot is None.

Source

pub fn speculative_supported(&self) -> bool

Whether the loaded model can self-speculate: a Qwen3.5 with a loaded (dense) MTP draft head. Pipeline/multi-device stage loads never own the MTP head, so this is false there too.

Source

pub fn forward_speculative( &self, input_ids: &[i64], start_pos: usize, last_logits_only: bool, ) -> Result<(Tensor, Tensor)>

Speculative prefill/verify forward: returns per-position logits plus the pre-final-norm hidden states the MTP head consumes.

Never delta-adapted: speculation is disabled for tenant-delta streams at stream construction, so this always runs the base model.

Returns (logits [1, seq|1, vocab], hidden [1, seq, hidden_size]).

Source

pub fn forward_mtp_draft( &self, token_ids: &[i64], hiddens: &Tensor, base: usize, ) -> Result<Tensor>

MTP draft forward over token_ids.len() consecutive slots. Slot i pairs hiddens[.., i, ..] with embed(token_ids[i]) at absolute position base + i. Returns the last slot’s logits [1, 1, vocab].

Source§

impl TorchEngine

Source

pub fn enable_training(&mut self, _learning_rate: f64) -> Result<()>

Enable training mode for pre-training

Source

pub fn manual_sgd_step(&mut self, _learning_rate: f32) -> Result<()>

Perform a manual SGD step without optimizer

Source

pub fn forward_training( &self, input_ids: &Tensor, track_gradients: bool, ) -> Result<Tensor>

Forward pass for training (with gradient tracking)

Source

pub fn compute_loss_and_backward( &self, input_ids: &Tensor, labels: &Tensor, ) -> Result<f64>

Compute loss and backward (without optimizer step)

Source

pub fn disable_training(&mut self) -> Result<()>

Disable training mode

Source

pub fn is_training_enabled(&self) -> bool

Check if training is enabled

Source§

impl TorchEngine

Source

pub fn is_persistent_model_ready(&self) -> bool

Check if persistent model is initialized - thread safe

Source

pub fn generate(&self, request: GenerationRequest) -> Result<TextStream<'_>>

Generate text as a stream of decoded UTF-8 text chunks

Returns a Stream that yields Result<String> with properly decoded text. The stream automatically handles:

  • Multi-byte UTF-8 sequences (emojis, CJK characters, etc.)
  • EOS and stop token detection
  • Max tokens limit
  • Special token filtering

Dropping the stream stops generation automatically (no manual cancellation needed).

§Example
use futures::StreamExt;
use hyprstream_rpc_std::inference_client::GenerationRequest;

let request = GenerationRequest::default();
let mut stream = engine.generate(request)?;

while let Some(text_chunk) = stream.next().await {
    print!("{}", text_chunk?);
}

let stats = stream.stats();
println!("Generated {} tokens", stats.tokens_generated);
Source

pub fn generate_with_delta( &self, request: GenerationRequest, delta: Option<Arc<Mutex<TenantDelta>>>, tenant_id: Option<String>, weight_epoch: u64, adapter_generation: u64, ) -> Result<TextStream<'_>>

Generate with optional per-tenant delta for delta-aware inference

When delta is Some, LoRA corrections are injected at each attention layer during generation. The delta is locked per-token (not held across the entire generation) to allow concurrent training updates.

adapter_generation is the live LoRA/delta generation actually applied to this request; it is folded into the KV-compat expected identity so a re-adapted model is detected as incompatible and its stale KV rejected.

tenant_id is the effective tenant whose delta contributed to the weights (Some only when a per-tenant delta is in play, None otherwise), and weight_epoch is that delta’s current weight epoch. The pair is threaded to the session-cache swap so the resulting KV is registered for tenant-delta invalidation and cleared if the weights changed under a different tenant or an older weight epoch (same-tenant in-place re-adaptation). The caller must derive the tenant id from the verified request subject, not invent a new identity source.

Source

pub async fn generate_with_delta_params( &self, request: GenerationRequest, delta: Option<Arc<Mutex<TenantDelta>>>, tenant_id: Option<String>, weight_epoch: u64, adapter_generation: u64, ) -> Result<GenerationResult>

Non-streaming generation with optional delta (convenience wrapper)

Source

pub fn forward_with_delta( &self, input_ids: &Tensor, delta: &TenantDelta, ) -> Result<Tensor>

Forward pass with per-layer delta injection for TTT training.

Uses decode_layer_with_delta() to inject the tenant delta’s A/B matrices after q_proj/v_proj projections inside each attention layer. This creates a differentiable path from the loss back to the delta parameters, enabling gradient-based training via loss.backward().

§Arguments
  • input_ids - Token IDs [1, seq_len]
  • delta - Tenant’s LoRA delta with trainable A/B matrices
§Returns

Logits tensor [1, seq_len, vocab_size] with gradient path to delta parameters

Source

pub fn effective_context_limit(&self) -> Result<usize>

Authoritative context capacity of the loaded model. ModelInfo is populated from the checkpoint configuration and optional max_context override during loading; RuntimeConfig.context_length is only a constructor default. Serving encoders and native decision admission must use this same limit.

Source

pub fn with_decision_backbone<T>( &self, token_ids: &[i64], head: impl FnOnce(&Tensor, &dyn Fn(&[i64]) -> Result<Tensor>) -> Result<T>, ) -> Result<T>

Execute a decision head against fresh full-sequence hidden states while holding the backbone lock. The supplied gather reads original LM-head rows without allocating token-by-vocabulary logits. No cache is reused.

Source

pub fn with_decision_backbone_checked<T>( &self, token_ids: &[i64], check: &dyn Fn() -> Result<()>, head: impl FnOnce(&Tensor, &dyn Fn(&[i64]) -> Result<Tensor>) -> Result<T>, ) -> Result<T>

Cooperative cancellation/deadline variant. The checker runs before admission, between decoder layers, and around head evaluation. An active libtorch kernel completes before the next check; dropping a future alone is not a compute-cancellation mechanism.

Source

pub fn get_num_layers(&self) -> Result<usize>

Get the number of layers in the loaded model.

Source

pub fn get_lora_module_dims(&self) -> Result<HashMap<String, (usize, usize)>>

Get module dimensions for LoRA target modules from the loaded model.

Returns a map of module_name -> (in_features, out_features) for all supported LoRA target modules. Used by DeltaPool to initialize per-tenant LoRA deltas with correct dimensions.

Supported modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Source

pub fn get_per_layer_lora_dims( &self, ) -> Option<HashMap<usize, HashMap<String, (usize, usize)>>>

Get per-layer LoRA module dimensions for hybrid architectures (e.g., Qwen3.5).

Returns None for uniform architectures (Llama, Gemma, etc.) where all layers have identical module dims. Returns per-layer overrides for Qwen3.5 where GDN layers have different o_proj input dims than full-attention layers.

Source

pub fn create_lora(&mut self, config: TenantDeltaConfig) -> Result<()>

Validate LoRA configuration against model dimensions.

Checks that the requested target modules are valid for this model. The actual LoRA delta creation is handled by DeltaPool/TenantDelta.

Source

pub fn extract_embedding(&self, text: &str) -> Result<Vec<f32>>

Extract embedding from text using the model’s hidden states.

This method tokenizes the input text, runs a forward pass through the model, and extracts the final hidden states. The embedding is computed by mean-pooling the hidden states over the sequence dimension.

§Arguments
  • text - Input text to embed
§Returns
  • Vec - The embedding vector (dimension = model’s hidden_size)
§Example
ⓘ
let embedding = engine.extract_embedding("What is machine learning?")?;
println!("Embedding dimension: {}", embedding.len());
Source

pub fn extract_embedding_from_tokens( &self, token_ids: &[i64], ) -> Result<Vec<f32>>

Extract embedding from pre-tokenized input.

This is a lower-level method for when tokens are already available (e.g., from a previous generation). Uses mean pooling over the hidden states.

§Arguments
  • token_ids - Pre-tokenized input IDs
§Returns
  • Vec - The embedding vector (dimension = model’s hidden_size)
Source

pub fn embed_images(&self, images: &[Vec<u8>]) -> Result<Vec<Vec<f32>>>

Compute vision embeddings for one or more images.

Preprocesses raw image bytes and runs them through the model’s vision encoder (e.g. SigLIP). Returns one embedding vector per image.

§Arguments
  • images - Raw image bytes (PNG/JPEG/RGB) per image
§Returns
  • Vec<Vec> - Embedding vectors, one per input image
Source

pub fn release_device_resources(&mut self)

Explicitly drop every CUDA/device tensor the engine owns or has cached on this thread, now — while the CUDA context is still alive.

§Why this exists (#429)

tch/libtorch defers CUDA context teardown to an atexit handler that runs during process exit. But two classes of device tensor outlive the engine struct and only drop later:

  1. Thread-local RoPE caches (llama::ROPE_CACHE, qwen3_5::ROPE_CACHE) — sin/cos tables cached on the compute device. They are TLS, so they are not touched by Drop and survive until the owning thread exits.
  2. The inference service thread itself — it is spawned detached in LocalInferenceService::start, so its TLS destructors (and thus the RoPE cache drop) can fire during process-exit cleanup, after the CUDA context is gone → c10::Error: invalid device pointer (SIGABRT).

The engine-only GPU smoke (e6_gpu_engine_is_single_device_on_one_gpu_host) never loads weights nor runs a forward pass, so it populates no RoPE cache and exits cleanly; the streaming test does, which is why only it aborts.

Calling this on the service thread at Shutdown receipt — before replying to the client — drops the RoPE tables and the model weights synchronously while the caller is still blocked on the reply (process indubitably alive, context valid). After it returns only empty shells remain for any late TLS/static destruction, so there is nothing left to free against a dead context.

§Contract

This is a terminal teardown: the engine is unusable afterward (model_info reports unloaded, generation will error). It must only be called once, at shutdown. Unlike Drop, it deliberately does clear shared Arc<Mutex<Option<T>>> contents — that is safe precisely because the engine is being retired, not because the contents are unused. (The production inference path holds a single engine instance per service; a cached clone would see None and fail closed with “No model loaded”, which is the correct behavior for a retired engine.)

Trait Implementations§

Source§

impl Clone for TorchEngine

Source§

fn clone(&self) -> TorchEngine

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Drop for TorchEngine

Source§

fn drop(&mut self)

Executes the destructor for this type. Read more
Source§

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)
Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned. Read more
Source§

impl RuntimeEngine for TorchEngine

Source§

fn load_model<'life0, 'life1, 'async_trait>( &'life0 mut self, path: &'life1 Path, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Load a model from the given path
Source§

fn generate<'life0, 'life1, 'async_trait>( &'life0 self, prompt: &'life1 str, max_tokens: usize, ) -> Pin<Box<dyn Future<Output = Result<String>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Generate text from a prompt (convenience method)
Source§

fn generate_with_params<'life0, 'async_trait>( &'life0 self, request: GenerationRequest, ) -> Pin<Box<dyn Future<Output = Result<GenerationResult>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait,

Generate text with full parameters (main method)
Source§

fn model_info(&self) -> ModelInfo

Get loaded model information
Source§

fn is_loaded(&self) -> bool

Check if model is loaded and ready
Source§

fn apply_chat_template( &self, messages: &[ChatMessage], add_generation_prompt: bool, tools: Option<&Value>, ) -> Result<String>

Apply chat template to messages (for template support). Read more
Source§

fn apply_chat_template_with_vars( &self, messages: &[ChatMessage], add_generation_prompt: bool, tools: Option<&Value>, enable_thinking: Option<bool>, template_vars_json: Option<&str>, ) -> Result<String>

Apply chat template with extended parameters (thinking mode, extra vars).

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
§

impl<T> Any for T
where T: Any,

§

fn into_any(self: Box<T>) -> Box<dyn Any>

§

fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>

§

fn type_name(&self) -> &'static str

§

impl<T> AnySync for T
where T: Any + Send + Sync,

§

fn into_any_arc(self: Arc<T>) -> Arc<dyn Any + Sync + Send> ⓘ

Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> FutureExt for T

§

fn with_context(self, otel_cx: Context) -> WithContext<Self>

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
§

fn with_current_context(self) -> WithContext<Self>

Attaches the current Context to this type, returning a WithContext wrapper. Read more
§

impl<T, W> HasTypeWitness<W> for T
where W: MakeTypeWitness<Arg = T>, T: ?Sized,

§

const WITNESS: W = W::MAKE

A constant of the type witness
§

impl<T> Identity for T
where T: ?Sized,

§

const TYPE_EQ: TypeEq<T, <T as Identity>::Type> = TypeEq::NEW

Proof that Self is the same type as Self::Type, provides methods for casting between Self and Self::Type.
§

type Type = T

The same type as Self, used to emulate type equality bounds (T == U) with associated type equality constraints (T: Identity<Type = U>).
§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
§

impl<T> Pointable for T

§

const ALIGN: usize

The alignment of pointer.
§

type Init = T

The type for initializers.
§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
§

impl<T> PolicyExt for T
where T: ?Sized,

§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] only if self and other return Action::Follow. Read more
§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] if either self or other returns Action::Follow. Read more
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
§

impl<T> ServiceExt for T

§

fn map_response_body<F>(self, f: F) -> MapResponseBody<Self, F>
where Self: Sized,

Apply a transformation to the response body. Read more
§

fn trace_for_http(self) -> Trace<Self, SharedClassifier<ServerErrorsAsFailures>>
where Self: Sized,

High level tracing that classifies responses using HTTP status codes. Read more
§

fn trace_for_grpc(self) -> Trace<Self, SharedClassifier<GrpcErrorsAsFailures>>
where Self: Sized,

High level tracing that classifies responses using gRPC headers. Read more
§

fn follow_redirects(self) -> FollowRedirect<Self>
where Self: Sized,

Follow redirect resposes using the Standard policy. Read more
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

impl<T> ErasedDestructor for T
where T: 'static,

§

impl<T> MaybeSend for T
where T: Send,

§

impl<T> MaybeSend for T
where T: Send,

§

impl<T> MaybeSync for T
where T: Sync,

§

impl<T> MaybeSync for T
where T: Sync,

§

impl<T> TerminalState for T
where T: Clone + Send + Sync,