pub struct KvCompatDescriptor {Show 35 fields
pub format_version: u32,
pub weights: WeightIdentity,
pub model_name: String,
pub architecture: String,
pub model_version: u32,
pub num_hidden_layers: usize,
pub num_attention_heads: usize,
pub num_key_value_heads: usize,
pub head_dim: usize,
pub hidden_size: usize,
pub use_qk_norm: bool,
pub partial_rotary_factor_bits: Option<u32>,
pub layer_types: Vec<String>,
pub linear_key_head_dim: usize,
pub linear_value_head_dim: usize,
pub linear_num_key_heads: usize,
pub linear_num_value_heads: usize,
pub linear_conv_kernel_dim: usize,
pub rms_norm_eps_bits: u32,
pub layer_norm_eps_bits: Option<u32>,
pub hidden_activation: String,
pub scale_embeddings: bool,
pub query_pre_attn_scalar_bits: Option<u32>,
pub moe: MoeRouting,
pub sliding_window: Option<usize>,
pub rope_local_base_freq_bits: Option<u32>,
pub rope_theta_bits: u32,
pub rope_scaling: Option<RopeScalingFp>,
pub max_position_embeddings: usize,
pub dtype: KvDtype,
pub kv_quant: KvQuantMode,
pub block_size: usize,
pub max_context: usize,
pub vocab_size: usize,
pub tokenizer_hash: Option<String>,
}Expand description
The complete set of authoritative inputs that determine whether KV states computed under one configuration can be reused under another.
Two caches are compatible iff their descriptors are equal in every
field. KvCompatFingerprint is the content-addressed digest of a canonical
serialization, so equality of fingerprints implies equality of descriptors
(modulo the astronomically unlikely SHA-256 collision). The descriptor is
Clone + Eq + Hash and small; the engine retains one (the “expected”
identity of the loaded model) and caches store a KvCompatFingerprint
stamped at first use.
Fields§
§format_version: u32Fingerprint format/layout version (KV_COMPAT_FORMAT_VERSION).
weights: WeightIdentityEffective weights (base + adapter/delta) identity.
model_name: StringLoaded model identity (path stem / name).
architecture: StringHuggingFace model_type (e.g. "llama", "qwen2").
model_version: u32Parsed config version.
num_attention_heads: usize§num_key_value_heads: usize§head_dim: usize§use_qk_norm: boolQK-norm applied to queries/keys (Gemma3, Qwen3 families). Off vs. on normalizes K and changes every downstream value.
partial_rotary_factor_bits: Option<u32>f32::to_bits of the partial-rotary factor. Determines rotary_dim
(head_dim * factor), i.e. how much of each key is rotated by RoPE —
a different factor rotates a different slice of K. None = full rotary.
layer_types: Vec<String>Per-layer attention type in layer order (e.g. "global"/"local",
"full_attention"/"linear_attention"). The local/global layout and any
local-RoPE behavior change which tokens each layer attends to and thus its
K/V. Stored lowercased; canonicalized at hash time.
linear_key_head_dim: usizeLinear/hybrid-attention state dimensions (Qwen3.5 GatedDeltaNet etc.). Non-zero on hybrid models; a different dimension changes the per-layer recurrent state that feeds later K/V.
linear_value_head_dim: usize§linear_num_key_heads: usize§linear_num_value_heads: usize§linear_conv_kernel_dim: usize§rms_norm_eps_bits: u32f32::to_bits of rms_norm_eps. The RMS epsilon changes the normalized
inputs to Q/K and every layer’s residual, so it changes all downstream
K/V. Stored as bits for byte-stable hashing (no f32 equality pitfalls).
layer_norm_eps_bits: Option<u32>f32::to_bits of layer_norm_eps when the architecture uses LayerNorm
instead of RMSNorm. None when LayerNorm is not in effect.
Activation function (e.g. "silu", "gelu_pytorch_tanh"). A different
activation changes every FFN output and thus the residual feeding later
K/V. Canonicalized (trimmed, lowercased) at construction.
scale_embeddings: boolWhether embeddings are scaled before entering the stack (changes all layer inputs and therefore all K/V).
query_pre_attn_scalar_bits: Option<u32>f32::to_bits of query_pre_attn_scalar (Gemma attention logit scale),
when set. Changes the attention scores and thus the per-layer output.
moe: MoeRoutingMoE routing config. is_moe == false and all-zero for a dense model.
sliding_window: Option<usize>Sliding-window size for local-attention layers (Gemma3), when set. Changes which tokens each local layer attends to and thus its K/V.
rope_local_base_freq_bits: Option<u32>f32::to_bits of rope_local_base_freq (the RoPE theta used in local
layers, Gemma3), when set. Changes local-layer post-RoPE key values.
rope_theta_bits: u32f32::to_bits of rope_theta.
rope_scaling: Option<RopeScalingFp>RoPE scaling, if any.
max_position_embeddings: usizeModel’s max_position_embeddings.
dtype: KvDtypeCompute dtype the model stores K/V in.
kv_quant: KvQuantModeKV storage encoding/quantization mode.
block_size: usizeTokens per paged block (must match for paged caches).
max_context: usizeEffective runtime context cap actually bounding the cache.
vocab_size: usize§tokenizer_hash: Option<String>Hex digest of the tokenizer bytes, when known.
Implementations§
Source§impl KvCompatDescriptor
impl KvCompatDescriptor
Sourcepub fn rope_theta(&self) -> f32
pub fn rope_theta(&self) -> f32
The RoPE theta, recovered from its bit pattern.
Sourcepub fn set_rope_theta(&mut self, theta: f32)
pub fn set_rope_theta(&mut self, theta: f32)
Set the RoPE theta from an f32 (canonicalized to bits).
Sourcepub fn partial_rotary_factor(&self) -> Option<f32>
pub fn partial_rotary_factor(&self) -> Option<f32>
The partial-rotary factor, recovered from its bit pattern (None = full rotary).
Sourcepub fn set_partial_rotary_factor(&mut self, factor: Option<f32>)
pub fn set_partial_rotary_factor(&mut self, factor: Option<f32>)
Set the partial-rotary factor from an f32 (canonicalized to bits).
Sourcepub fn set_rms_norm_eps(&mut self, eps: f32)
pub fn set_rms_norm_eps(&mut self, eps: f32)
Set the RMS norm epsilon from an f32 (canonicalized to bits).
Sourcepub fn set_layer_norm_eps(&mut self, eps: Option<f32>)
pub fn set_layer_norm_eps(&mut self, eps: Option<f32>)
Set the LayerNorm epsilon from an f32, or None.
Sourcepub fn set_query_pre_attn_scalar(&mut self, scalar: Option<f32>)
pub fn set_query_pre_attn_scalar(&mut self, scalar: Option<f32>)
Set the query pre-attention scalar from an f32, or None.
Sourcepub fn set_rope_local_base_freq(&mut self, freq: Option<f32>)
pub fn set_rope_local_base_freq(&mut self, freq: Option<f32>)
Set the local-RoPE base frequency from an f32, or None.
True iff every authoritative identity axis is populated.
A descriptor missing the base-weight digest or the tokenizer identity was built from a non-authoritative source and must not be used to bless a cache for reuse: the engine treats a non-authoritative expected identity as fail-closed (discard + recompute) rather than stamping a guess.
Sourcepub fn set_tokenizer(
&mut self,
vocab_size: usize,
hash: impl Into<Option<String>>,
)
pub fn set_tokenizer( &mut self, vocab_size: usize, hash: impl Into<Option<String>>, )
Record the tokenizer identity (vocab size + digest of its bytes).
Called by the engine once the tokenizer is loaded; the descriptor is finalized before any generation reads its fingerprint.
Sourcepub fn fingerprint(&self) -> KvCompatFingerprint
pub fn fingerprint(&self) -> KvCompatFingerprint
Content-addressed fingerprint of this descriptor.
Sourcepub fn check_against(
&self,
observed: &KvCompatDescriptor,
) -> Result<(), KvCompatMismatch>
pub fn check_against( &self, observed: &KvCompatDescriptor, ) -> Result<(), KvCompatMismatch>
Rejection policy against an observed (cache-side) descriptor.
Returns the first differing field in a fixed order, or Ok(()) if the
descriptors are fully compatible. See check_compatibility.
Trait Implementations§
Source§impl Clone for KvCompatDescriptor
impl Clone for KvCompatDescriptor
Source§fn clone(&self) -> KvCompatDescriptor
fn clone(&self) -> KvCompatDescriptor
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for KvCompatDescriptor
impl Debug for KvCompatDescriptor
Source§impl Default for KvCompatDescriptor
impl Default for KvCompatDescriptor
Source§fn default() -> KvCompatDescriptor
fn default() -> KvCompatDescriptor
Source§impl Hash for KvCompatDescriptor
impl Hash for KvCompatDescriptor
Source§impl PartialEq for KvCompatDescriptor
impl PartialEq for KvCompatDescriptor
Source§fn eq(&self, other: &KvCompatDescriptor) -> bool
fn eq(&self, other: &KvCompatDescriptor) -> bool
self and other values to be equal, and is used by ==.impl Eq for KvCompatDescriptor
impl StructuralPartialEq for KvCompatDescriptor
Auto Trait Implementations§
impl Freeze for KvCompatDescriptor
impl RefUnwindSafe for KvCompatDescriptor
impl Send for KvCompatDescriptor
impl Sync for KvCompatDescriptor
impl Unpin for KvCompatDescriptor
impl UnsafeUnpin for KvCompatDescriptor
impl UnwindSafe for KvCompatDescriptor
Blanket Implementations§
§impl<T> AnySync for T
impl<T> AnySync for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
§impl<T> CallHasher for T
impl<T> CallHasher for T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key and return true if they are equal.§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
§impl<T> FutureExt for T
impl<T> FutureExt for T
§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more