Skip to main content

KvCompatDescriptor

Struct KvCompatDescriptor 

Source
pub struct KvCompatDescriptor {
Show 35 fields pub format_version: u32, pub weights: WeightIdentity, pub model_name: String, pub architecture: String, pub model_version: u32, pub num_hidden_layers: usize, pub num_attention_heads: usize, pub num_key_value_heads: usize, pub head_dim: usize, pub hidden_size: usize, pub use_qk_norm: bool, pub partial_rotary_factor_bits: Option<u32>, pub layer_types: Vec<String>, pub linear_key_head_dim: usize, pub linear_value_head_dim: usize, pub linear_num_key_heads: usize, pub linear_num_value_heads: usize, pub linear_conv_kernel_dim: usize, pub rms_norm_eps_bits: u32, pub layer_norm_eps_bits: Option<u32>, pub hidden_activation: String, pub scale_embeddings: bool, pub query_pre_attn_scalar_bits: Option<u32>, pub moe: MoeRouting, pub sliding_window: Option<usize>, pub rope_local_base_freq_bits: Option<u32>, pub rope_theta_bits: u32, pub rope_scaling: Option<RopeScalingFp>, pub max_position_embeddings: usize, pub dtype: KvDtype, pub kv_quant: KvQuantMode, pub block_size: usize, pub max_context: usize, pub vocab_size: usize, pub tokenizer_hash: Option<String>,
}
Expand description

The complete set of authoritative inputs that determine whether KV states computed under one configuration can be reused under another.

Two caches are compatible iff their descriptors are equal in every field. KvCompatFingerprint is the content-addressed digest of a canonical serialization, so equality of fingerprints implies equality of descriptors (modulo the astronomically unlikely SHA-256 collision). The descriptor is Clone + Eq + Hash and small; the engine retains one (the “expected” identity of the loaded model) and caches store a KvCompatFingerprint stamped at first use.

Fields§

§format_version: u32

Fingerprint format/layout version (KV_COMPAT_FORMAT_VERSION).

§weights: WeightIdentity

Effective weights (base + adapter/delta) identity.

§model_name: String

Loaded model identity (path stem / name).

§architecture: String

HuggingFace model_type (e.g. "llama", "qwen2").

§model_version: u32

Parsed config version.

§num_hidden_layers: usize§num_attention_heads: usize§num_key_value_heads: usize§head_dim: usize§hidden_size: usize§use_qk_norm: bool

QK-norm applied to queries/keys (Gemma3, Qwen3 families). Off vs. on normalizes K and changes every downstream value.

§partial_rotary_factor_bits: Option<u32>

f32::to_bits of the partial-rotary factor. Determines rotary_dim (head_dim * factor), i.e. how much of each key is rotated by RoPE — a different factor rotates a different slice of K. None = full rotary.

§layer_types: Vec<String>

Per-layer attention type in layer order (e.g. "global"/"local", "full_attention"/"linear_attention"). The local/global layout and any local-RoPE behavior change which tokens each layer attends to and thus its K/V. Stored lowercased; canonicalized at hash time.

§linear_key_head_dim: usize

Linear/hybrid-attention state dimensions (Qwen3.5 GatedDeltaNet etc.). Non-zero on hybrid models; a different dimension changes the per-layer recurrent state that feeds later K/V.

§linear_value_head_dim: usize§linear_num_key_heads: usize§linear_num_value_heads: usize§linear_conv_kernel_dim: usize§rms_norm_eps_bits: u32

f32::to_bits of rms_norm_eps. The RMS epsilon changes the normalized inputs to Q/K and every layer’s residual, so it changes all downstream K/V. Stored as bits for byte-stable hashing (no f32 equality pitfalls).

§layer_norm_eps_bits: Option<u32>

f32::to_bits of layer_norm_eps when the architecture uses LayerNorm instead of RMSNorm. None when LayerNorm is not in effect.

§hidden_activation: String

Activation function (e.g. "silu", "gelu_pytorch_tanh"). A different activation changes every FFN output and thus the residual feeding later K/V. Canonicalized (trimmed, lowercased) at construction.

§scale_embeddings: bool

Whether embeddings are scaled before entering the stack (changes all layer inputs and therefore all K/V).

§query_pre_attn_scalar_bits: Option<u32>

f32::to_bits of query_pre_attn_scalar (Gemma attention logit scale), when set. Changes the attention scores and thus the per-layer output.

§moe: MoeRouting

MoE routing config. is_moe == false and all-zero for a dense model.

§sliding_window: Option<usize>

Sliding-window size for local-attention layers (Gemma3), when set. Changes which tokens each local layer attends to and thus its K/V.

§rope_local_base_freq_bits: Option<u32>

f32::to_bits of rope_local_base_freq (the RoPE theta used in local layers, Gemma3), when set. Changes local-layer post-RoPE key values.

§rope_theta_bits: u32

f32::to_bits of rope_theta.

§rope_scaling: Option<RopeScalingFp>

RoPE scaling, if any.

§max_position_embeddings: usize

Model’s max_position_embeddings.

§dtype: KvDtype

Compute dtype the model stores K/V in.

§kv_quant: KvQuantMode

KV storage encoding/quantization mode.

§block_size: usize

Tokens per paged block (must match for paged caches).

§max_context: usize

Effective runtime context cap actually bounding the cache.

§vocab_size: usize§tokenizer_hash: Option<String>

Hex digest of the tokenizer bytes, when known.

Implementations§

Source§

impl KvCompatDescriptor

Source

pub fn rope_theta(&self) -> f32

The RoPE theta, recovered from its bit pattern.

Source

pub fn set_rope_theta(&mut self, theta: f32)

Set the RoPE theta from an f32 (canonicalized to bits).

Source

pub fn partial_rotary_factor(&self) -> Option<f32>

The partial-rotary factor, recovered from its bit pattern (None = full rotary).

Source

pub fn set_partial_rotary_factor(&mut self, factor: Option<f32>)

Set the partial-rotary factor from an f32 (canonicalized to bits).

Source

pub fn set_rms_norm_eps(&mut self, eps: f32)

Set the RMS norm epsilon from an f32 (canonicalized to bits).

Source

pub fn set_layer_norm_eps(&mut self, eps: Option<f32>)

Set the LayerNorm epsilon from an f32, or None.

Source

pub fn set_query_pre_attn_scalar(&mut self, scalar: Option<f32>)

Set the query pre-attention scalar from an f32, or None.

Source

pub fn set_rope_local_base_freq(&mut self, freq: Option<f32>)

Set the local-RoPE base frequency from an f32, or None.

Source

pub fn is_authoritatively_complete(&self) -> bool

True iff every authoritative identity axis is populated.

A descriptor missing the base-weight digest or the tokenizer identity was built from a non-authoritative source and must not be used to bless a cache for reuse: the engine treats a non-authoritative expected identity as fail-closed (discard + recompute) rather than stamping a guess.

Source

pub fn set_tokenizer( &mut self, vocab_size: usize, hash: impl Into<Option<String>>, )

Record the tokenizer identity (vocab size + digest of its bytes).

Called by the engine once the tokenizer is loaded; the descriptor is finalized before any generation reads its fingerprint.

Source

pub fn fingerprint(&self) -> KvCompatFingerprint

Content-addressed fingerprint of this descriptor.

Source

pub fn check_against( &self, observed: &KvCompatDescriptor, ) -> Result<(), KvCompatMismatch>

Rejection policy against an observed (cache-side) descriptor.

Returns the first differing field in a fixed order, or Ok(()) if the descriptors are fully compatible. See check_compatibility.

Trait Implementations§

Source§

impl Clone for KvCompatDescriptor

Source§

fn clone(&self) -> KvCompatDescriptor

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for KvCompatDescriptor

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more
Source§

impl Default for KvCompatDescriptor

Source§

fn default() -> KvCompatDescriptor

Returns the “default value” for a type. Read more
Source§

impl Hash for KvCompatDescriptor

Source§

fn hash<__H>(&self, state: &mut __H)
where __H: Hasher,

Feeds this value into the given Hasher. Read more
1.3.0 · Source§

fn hash_slice<H>(data: &[Self], state: &mut H)
where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher. Read more
Source§

impl PartialEq for KvCompatDescriptor

Source§

fn eq(&self, other: &KvCompatDescriptor) -> bool

Tests for self and other values to be equal, and is used by ==.
1.0.0 (const: unstable) · Source§

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.
Source§

impl Eq for KvCompatDescriptor

Source§

impl StructuralPartialEq for KvCompatDescriptor

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
§

impl<T> Any for T
where T: Any,

§

fn into_any(self: Box<T>) -> Box<dyn Any>

§

fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>

§

fn type_name(&self) -> &'static str

§

impl<T> AnySync for T
where T: Any + Send + Sync,

§

fn into_any_arc(self: Arc<T>) -> Arc<dyn Any + Sync + Send> ⓘ

§

impl<'a, T, E> AsTaggedExplicit<'a, E> for T
where T: 'a,

§

fn explicit(self, class: Class, tag: u32) -> TaggedParser<'a, Explicit, Self, E>

§

impl<'a, T, E> AsTaggedImplicit<'a, E> for T
where T: 'a,

§

fn implicit( self, class: Class, constructed: bool, tag: u32, ) -> TaggedParser<'a, Implicit, Self, E>

Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
§

impl<T> CallHasher for T
where T: Hash + ?Sized,

§

fn get_hash<H, B>(value: &H, build_hasher: &B) -> u64
where H: Hash + ?Sized, B: BuildHasher,

Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> DynClone for T
where T: Clone,

Source§

fn __clone_box(&self, _: Private) -> *mut ()

§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

§

fn equivalent(&self, key: &K) -> bool

Compare self to key and return true if they are equal.
§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

§

fn equivalent(&self, key: &K) -> bool

Checks if this value is equivalent to the given key. Read more
Source§

impl<Q, K> Equivalent<K> for Q
where Q: Eq + ?Sized, K: Borrow<Q> + ?Sized,

Source§

fn equivalent(&self, key: &K) -> bool

Compare self to key and return true if they are equal.
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> FromRef<T> for T
where T: Clone,

§

fn from_ref(input: &T) -> T

Converts to this type from a reference to the input type.
§

impl<T> FutureExt for T

§

fn with_context(self, otel_cx: Context) -> WithContext<Self>

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
§

fn with_current_context(self) -> WithContext<Self>

Attaches the current Context to this type, returning a WithContext wrapper. Read more
§

impl<T, W> HasTypeWitness<W> for T
where W: MakeTypeWitness<Arg = T>, T: ?Sized,

§

const WITNESS: W = W::MAKE

A constant of the type witness
§

impl<T> Identity for T
where T: ?Sized,

§

const TYPE_EQ: TypeEq<T, <T as Identity>::Type> = TypeEq::NEW

Proof that Self is the same type as Self::Type, provides methods for casting between Self and Self::Type.
§

type Type = T

The same type as Self, used to emulate type equality bounds (T == U) with associated type equality constraints (T: Identity<Type = U>).
§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more
§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> IntoRequest<T> for T

Source§

fn into_request(self) -> Request<T>

Wrap the input message T in a tonic::Request
§

impl<T> JaggedSlice<T> for T

§

type Index = Index2

§

fn push_into(self, array: &mut Jagged<T>)

§

fn insert_into(self, index: <T as JaggedSlice<T>>::Index, array: &mut Jagged<T>)

§

impl<T> NoneValue for T
where T: Default,

§

type NoneType = T

§

fn null_value() -> T

The none-equivalent value.
§

impl<T> NoneValue for T
where T: Default,

§

type NoneType = T

§

fn null_value() -> T

The none-equivalent value.
§

impl<T> Pointable for T

§

const ALIGN: usize

The alignment of pointer.
§

type Init = T

The type for initializers.
§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
§

impl<T> PolicyExt for T
where T: ?Sized,

§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] only if self and other return Action::Follow. Read more
§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns [Action::Follow] if either self or other returns Action::Follow. Read more
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
§

impl<T> ServiceExt for T

§

fn map_response_body<F>(self, f: F) -> MapResponseBody<Self, F>
where Self: Sized,

Apply a transformation to the response body. Read more
§

fn trace_for_http(self) -> Trace<Self, SharedClassifier<ServerErrorsAsFailures>>
where Self: Sized,

High level tracing that classifies responses using HTTP status codes. Read more
§

fn trace_for_grpc(self) -> Trace<Self, SharedClassifier<GrpcErrorsAsFailures>>
where Self: Sized,

High level tracing that classifies responses using gRPC headers. Read more
§

fn follow_redirects(self) -> FollowRedirect<Self>
where Self: Sized,

Follow redirect resposes using the Standard policy. Read more
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

§

fn vzip(self) -> V

§

impl<T> WithSubscriber for T

§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a [WithDispatch] wrapper. Read more
§

impl<T> ErasedDestructor for T
where T: 'static,

§

impl<T> EventKey for T
where T: Hash + PartialEq + Eq + Send + Sync,

§

impl<T> MaybeSend for T
where T: Send,

§

impl<T> MaybeSend for T
where T: Send,

§

impl<T> MaybeSync for T
where T: Sync,

§

impl<T> MaybeSync for T
where T: Sync,

Source§

impl<T> ResourceKey for T
where T: Eq + Hash + Clone + Send + Sync + Debug,

§

impl<E> ResultError for E
where E: Send + Debug + Sync,

§

impl<T> ResultType for T
where T: Send + Clone + Sync + Debug,

§

impl<T> SendSyncUnwindSafe for T
where T: Send + Sync + UnwindSafe + ?Sized,

Source§

impl<T> TerminalState for T
where T: Clone + Send + Sync,