hyprstream_core/storage/cas/mod.rs
1//! L1 unified content-addressed store (CAS) substrate — #812 (epic #809, W1).
2//!
3//! One content-addressed store under everything. Every artifact is addressed by a
4//! self-describing **multihash** ([`hyprstream_rpc::cid`]), and every ingest is
5//! partitioned into a **dedup domain** keyed `(compartment, algorithm,
6//! digest_length, trust_boundary)` so content is only ever deduplicated against
7//! other content in the *same* domain.
8//!
9//! ## What this layer is
10//!
11//! [`CasSubstrate`] absorbs [`cas_serve::CasStore`] (the write/reconstruct core
12//! left behind the retired `cas-serve` binary) behind a domain-partitioned,
13//! multihash-addressed facade. The registry `putBlob`/`getBlob` paths and the XET
14//! `get_xorb` route reach the store *through* the substrate now, never a bare
15//! `CasStore::from_env()`.
16//!
17//! - **Chunking** is unchanged: gearhash CDC via `cas_serve::chunker`, byte-for-byte
18//! identical to xet-core (see that crate's `chunker.rs` golden test).
19//! - **Default algorithm** is BLAKE3-256, aligning with the existing
20//! merklehash/xet substrate.
21//! - **On-disk layout** for the *default* domain `(None, Blake3, Local)` is
22//! identical to the legacy `CasStore` root, so existing stores and callers keep
23//! working byte-for-byte. Non-default domains get their own sub-root.
24//!
25//! ## Dedup domains ([`DedupDomain`])
26//!
27//! Content in different domains is **never** deduplicated together, because each
28//! domain is a distinct storage sub-root:
29//!
30//! - **compartment** — tenant / need-to-know isolation, keyed off the existing MAC
31//! [`Compartment`](hyprstream_rpc::auth::mac::Compartment) vocabulary
32//! (READ-only here — the substrate never authors or enforces MAC policy).
33//! - **algorithm** — SHA-1 content is never deduplicated against SHA-256; the pair
34//! would lean on the weaker collision resistance.
35//! - **digest_length** — length is part of multihash/CID identity, so it is part of
36//! the dedup key: BLAKE3 is an XOF and `BLAKE3-512(x)[0:32] == BLAKE3-256(x)`,
37//! so the two widths must not share a storage root (#812/#881 joint decision).
38//! Ingest is BLAKE3-256 today; a 512-bit width lands as its own root in #881.
39//! - **trust_boundary** — node-local content is never deduplicated against
40//! shared-remote content (an existence/content oracle across the boundary, U5).
41//!
42//! ## MAC boundary
43//!
44//! A [`BlobManifest`] is a sealed, labeled object: its required
45//! `security_label` is part of the bytes hashed into its CID. Staging may be
46//! unlabeled, but it cannot seal until a label has been supplied.
47
48mod domain;
49mod manifest;
50mod mount;
51mod substrate;
52
53pub use domain::{DedupDomain, InvalidDedupDomain, TrustBoundary};
54pub use manifest::{cid_from_merkle, merkle_from_address, BlobManifest, FILE_RECONSTRUCTION_CODEC};
55pub use mount::{
56 AllowAllCasAuthorizer, BootstrapCasAuthorizer, CasGrantSubject, CasMount, CasMountAuthorizer,
57 CasMountAuthzRequest, CasMountGrant, CasMountObjectKind, DenyAllCasAuthorizer, StagingConfig,
58 AUTHENTICATED_XORB_READ, DEFAULT_STAGING_SLOT_QUOTA, DEFAULT_STAGING_SUBJECT_QUOTA,
59};
60pub use substrate::CasSubstrate;
61
62/// Errors from the L1 CAS substrate.
63#[derive(Debug, thiserror::Error)]
64pub enum CasError {
65 /// The underlying content-addressed store failed (io, not-found, corrupt shard).
66 #[error("cas store: {0}")]
67 Store(#[from] cas_serve::StoreError),
68
69 /// A content address (CID / multihash) could not be encoded or decoded.
70 #[error("content address: {0}")]
71 Cid(String),
72
73 /// A hex digest was malformed.
74 #[error("invalid hex digest: {0}")]
75 Hex(String),
76
77 /// A persisted sealed manifest could not be read, decoded, or verified.
78 #[error("CAS manifest: {0}")]
79 Manifest(String),
80
81 /// Ingest was requested for an algorithm the substrate cannot currently
82 /// *produce*. The domain vocabulary supports every algorithm for keying, but
83 /// the ingest path only computes BLAKE3-256 today (see [`CasSubstrate::put`]).
84 #[error("unsupported ingest algorithm: {0:?} (only BLAKE3-256 ingest is implemented)")]
85 UnsupportedIngestAlgorithm(hyprstream_rpc::cid::HashAlgo),
86}