pub enum AttentionType {
MHA,
MQA,
GQA {
num_groups: usize,
},
}Expand description
Attention mechanism types
Variants§
MHA
Multi-Head Attention (all heads have separate K,V)
MQA
Multi-Query Attention (single K,V shared across all Q heads)
GQA
Grouped-Query Attention (groups of Q heads share K,V)
Implementations§
Source§impl AttentionType
impl AttentionType
Sourcepub fn kv_expansion_factor(&self, num_heads: usize) -> usize
pub fn kv_expansion_factor(&self, num_heads: usize) -> usize
Get the expansion factor for key-value heads
Sourcepub fn supports_kv_cache_sharing(&self) -> bool
pub fn supports_kv_cache_sharing(&self) -> bool
Check if KV cache can be shared
Trait Implementations§
Source§impl Clone for AttentionType
impl Clone for AttentionType
Source§fn clone(&self) -> AttentionType
fn clone(&self) -> AttentionType
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreSource§impl Debug for AttentionType
impl Debug for AttentionType
Source§impl PartialEq for AttentionType
impl PartialEq for AttentionType
Source§fn eq(&self, other: &AttentionType) -> bool
fn eq(&self, other: &AttentionType) -> bool
Tests for
self and other values to be equal, and is used by ==.impl StructuralPartialEq for AttentionType
Auto Trait Implementations§
impl Freeze for AttentionType
impl RefUnwindSafe for AttentionType
impl Send for AttentionType
impl Sync for AttentionType
impl Unpin for AttentionType
impl UnsafeUnpin for AttentionType
impl UnwindSafe for AttentionType
Blanket Implementations§
§impl<T> AnySync for T
impl<T> AnySync for T
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
§impl<T> FutureExt for T
impl<T> FutureExt for T
§fn with_context(self, otel_cx: Context) -> WithContext<Self>
fn with_context(self, otel_cx: Context) -> WithContext<Self>
§fn with_current_context(self) -> WithContext<Self>
fn with_current_context(self) -> WithContext<Self>
§impl<T> Identity for Twhere
T: ?Sized,
impl<T> Identity for Twhere
T: ?Sized,
§impl<T> Instrument for T
impl<T> Instrument for T
§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more§impl<T> Pointable for T
impl<T> Pointable for T
§impl<T> PolicyExt for Twhere
T: ?Sized,
impl<T> PolicyExt for Twhere
T: ?Sized,
§impl<T> ServiceExt for T
impl<T> ServiceExt for T
§fn map_response_body<F>(self, f: F) -> MapResponseBody<Self, F>where
Self: Sized,
fn map_response_body<F>(self, f: F) -> MapResponseBody<Self, F>where
Self: Sized,
Apply a transformation to the response body. Read more
§fn trace_for_http(self) -> Trace<Self, SharedClassifier<ServerErrorsAsFailures>>where
Self: Sized,
fn trace_for_http(self) -> Trace<Self, SharedClassifier<ServerErrorsAsFailures>>where
Self: Sized,
High level tracing that classifies responses using HTTP status codes. Read more
§fn trace_for_grpc(self) -> Trace<Self, SharedClassifier<GrpcErrorsAsFailures>>where
Self: Sized,
fn trace_for_grpc(self) -> Trace<Self, SharedClassifier<GrpcErrorsAsFailures>>where
Self: Sized,
High level tracing that classifies responses using gRPC headers. Read more