Skip to main content

Module delta_pool

Module delta_pool 

Source
Expand description

Per-tenant delta registry with LRU eviction

Manages a pool of TenantDelta instances, one per tenant/session. Pattern mirrors KVCacheRegistry using DashMap for concurrent access.

§Snapshot durability & lifecycle (#869)

Uncommitted per-tenant TTT adaptation is persisted to adapters/.snapshots/{subject}.safetensors on two occasions:

The read half is DeltaPool::get_or_hydrate, which reloads a snapshot before zero-initializing a fresh delta, restoring weights, effective ranks, Muon momentum, and accumulated_steps / request_count.

Lifecycle decision — keep, don’t delete on load. A rehydrated snapshot is left on disk after a successful load. Deleting it on load would open a window where a crash between hydrate and the next eviction/export loses every step accumulated since — the exact failure #869 exists to close. The file is instead overwritten by the next eviction/export snapshot of the same subject, so at worst a reader sees slightly stale (never absent) state. These staging snapshots stay in .snapshots/ and are never auto-promoted into a committed adapters/NN_*.safetensors adapter — that crossing is the STEP Promote boundary and remains an explicit operation.

What a snapshot does NOT preserve (rehydrated deltas start blank for these; see TenantDelta::extract_state_dict): the in-memory STEP Pending rollback state (DeltaAdaptationState::Pending) — a speculative, not-yet-resolved adaptation is folded into the committed weights it was applied on top of, so it survives as committed rather than as a reversible pending. No SSM/optimizer state beyond Muon momentum is carried (there is none in TenantDelta today). avg_loss_improvement and last_snapshot_hash reset.

Structs§

DeltaPool
Registry managing per-tenant LoRA deltas for isolated TTT adaptation