Expand description
Torch inference engine and training pipeline for hyprstream.
Carved out of the hyprstream application crate (Wave B decomposition) so
that libtorch links only into binaries that actually run inference or
training. Everything here lives behind the engine trait
runtime::RuntimeEngine, whose surface carries only
hyprstream-rpc-std types — no tch types leak across the crate boundary,
and callers below this crate consume the engine through that trait.
runtime— PyTorch engine (torch_engine), model architectures, model factory, KV cache, sampling, chat templates, multi-GPU device pooltraining— Test-Time Training (TTT), per-tenant LoRA deltas (tenant_delta,delta_pool), merge strategies, quality filter, checkpointsconfig— the inference-facing configuration surface (RuntimeConfig,ModelConfig, generation/training configs); re-exported by the main crate’sconfigmodule so existing paths keep resolvingworktree_ext— 9P worktree-client convenience helpers and the COW-aware checkpoint copy used by the training pipeline
tch tensors are !Send; engine services that hold them spawn through the
hyprstream-service Thread spawner (the existing Spawnable !Send impl
path). This crate never forces Send across the inference boundary.
License: AGPL-3.0-only. This crate is carved from the AGPL hyprstream
application and is part of its combined AGPL distribution.
Modules§
- config
- Inference-facing configuration types.
- runtime
- Runtime abstraction layer for inference engines
- training
- Training utilities and checkpoint management
- worktree_
ext - 9P convenience helpers for WorktreeClient.