Skip to main content

hyprstream_inference/
lib.rs

1//! Torch inference engine and training pipeline for hyprstream.
2//!
3//! Carved out of the `hyprstream` application crate (Wave B decomposition) so
4//! that libtorch links only into binaries that actually run inference or
5//! training. Everything here lives behind the engine trait
6//! [`runtime::RuntimeEngine`], whose surface carries only
7//! `hyprstream-rpc-std` types — no `tch` types leak across the crate boundary,
8//! and callers below this crate consume the engine through that trait.
9//!
10//! - `runtime` — PyTorch engine (`torch_engine`), model architectures, model
11//!   factory, KV cache, sampling, chat templates, multi-GPU device pool
12//! - `training` — Test-Time Training (TTT), per-tenant LoRA deltas
13//!   (`tenant_delta`, `delta_pool`), merge strategies, quality filter,
14//!   checkpoints
15//! - `config` — the inference-facing configuration surface (`RuntimeConfig`,
16//!   `ModelConfig`, generation/training configs); re-exported by the main
17//!   crate's `config` module so existing paths keep resolving
18//! - `worktree_ext` — 9P worktree-client convenience helpers and the
19//!   COW-aware checkpoint copy used by the training pipeline
20//!
21//! `tch` tensors are `!Send`; engine services that hold them spawn through the
22//! `hyprstream-service` Thread spawner (the existing `Spawnable` !Send impl
23//! path). This crate never forces `Send` across the inference boundary.
24//!
25//! License: AGPL-3.0-only. This crate is carved from the AGPL `hyprstream`
26//! application and is part of its combined AGPL distribution.
27
28pub mod config;
29pub mod runtime;
30pub mod training;
31pub mod worktree_ext;