Skip to main content

hyprstream_discovery/
admission_roster.rs

1//! Derived native admission roster (#1652): deployment service identities
2//! resolved from the live accepted-state store at admission time, not from
3//! DID-valued process configuration.
4//!
5//! Formerly `quic.moql_subject_tenants` embedded every deployment DID in
6//! config.toml, so each service-identity re-initialization (R1: reviewed
7//! store wipe + re-provision, all DIDs change) left every running process
8//! pinned to stale DIDs until the boot projector re-wrote the file AND every
9//! container restarted. The tenant is a pure roster projection — metal writes
10//! `did = "local"` uniformly for all eight services — so it is now DERIVED:
11//! DID → accepted state → capsule `#<name>` service entry ∩ the registered
12//! factory roster, held UNIQUELY across the store ⇒ tenant [`LOCAL_TENANT`];
13//! everything else ⇒ unresolved, deny.
14//!
15//! # Why unique-holder, not name-match alone
16//!
17//! The store is not a pure deployment roster: the public `ingestAt9pCandidate`
18//! RPC admits foreign, self-certifying capsules whose `#id` entries are
19//! arbitrary strings, and provisioned and ingested states share one write
20//! path — provenance is not distinguishable in the store. A foreign capsule
21//! that NAMES a factory service (e.g. `#event`) therefore cannot be told
22//! apart from a deployment identity by its content. The derivation closes
23//! that self-selection the only way store data allows:
24//!
25//! - a name admits a tenant only while EXACTLY ONE accepted state in the
26//!   whole store derives it — beside the genuine identity a squatter makes
27//!   the name ambiguous, and ambiguity denies (fail-closed, loud: the
28//!   collision is warned so an operator sees the injected record). This is
29//!   the offline roster validator's cross-state uniqueness `ensure!`, and
30//!   `project_bootstrap_endpoint`'s ambiguity rule, applied per lookup;
31//! - the deriving state must carry the deployment invariants the offline
32//!   provisioner guarantees and foreign genesis-only records lack: successor
33//!   epoch (`epoch > 0`) and a bounded expiry that has not lapsed (an expired
34//!   state — genuine or foreign — never derives and never counts toward
35//!   uniqueness).
36//!
37//! Residual (deliberate, flagged in #1652): if a deployment name has NO
38//! genuine accepted state (decommissioned service) and an attacker ingests a
39//! capsule claiming it, that name derives for the attacker. Operating with
40//! names in `quic.event_publishers` that have no live deployment state is a
41//! deployment inconsistency this PR refuses at Event spawn
42//! ([`require_publisher_roster`]) and the tenant grant remains bounded to the
43//! fixed local namespace. Fully distinguishing provenance needs a
44//! provisioner-signed roster the admission path can verify — a maintainer
45//! design decision recorded in #1652, not something this change invents.
46//!
47//! Resolution enumerates the store's verified states through the same
48//! verification as the per-DID admission read, behind a TTL cache
49//! ([`CachedRoster`]): the live-session recheck runs every 100 ms per session,
50//! so the polling path serves a bounded-stale snapshot instead of re-verifying
51//! the whole store at 10 Hz per session. Revocation by state REMOVAL stays
52//! immediate (the per-DID admission authority recheck is uncached); the cache
53//! bounds only how quickly roster-side changes — churn re-binding, name
54//! reassignment, squatter arrival — become visible (≤ TTL).
55//!
56//! This mirrors the hosted-account tenant index
57//! (`HostedAccountStore::resolve_tenant_for_hosted_did`) and the
58//! `MoqConnectAuthz` resolver-closure shape: the tenant never comes from the
59//! peer, its proof, or a token — only server-side resolution over verified
60//! accepted state. Failure inheritance is identical to authentication: a
61//! store read failure yields an empty roster, and an empty roster denies.
62
63use anyhow::Result;
64use hyprstream_rpc::moq_authz::PeerIdentity;
65use hyprstream_rpc::transport::iroh_moq::PeerTenantResolver;
66use hyprstream_rpc::transport::moql_admission::{AcceptedIdentityState, MoqlAdmissionProof};
67use std::sync::Arc;
68
69/// The single tenant of the fixed native Event/Streams namespace. The Event
70/// tree is the fixed local namespace (`local/events`); every deployment
71/// service identity maps to it and no other tenant is derivable.
72pub const LOCAL_TENANT: &str = "local";
73
74/// One projected accepted state, keyed by its DID: the roster universe the
75/// derivation enumerates (provisioned deployment identities and foreign
76/// ingested records alike).
77pub type RosterEntry = (String, AcceptedIdentityState);
78
79/// The deployment-roster side of the accepted-state authority: enumerates
80/// every verified accepted state so grant derivation can enforce the
81/// cross-store uniqueness a per-DID read cannot see. Implemented by the
82/// process's cached projection; a blanket `Fn` impl covers fixtures.
83///
84/// The snapshot is shared as an `Arc`: the 100 ms live-session recheck must
85/// not deep-copy the roster (ML-DSA key buffers included) per poll —
86/// consumers borrow the slice.
87pub trait DeploymentRosterSource: Send + Sync {
88    fn roster(&self) -> Arc<Vec<RosterEntry>>;
89}
90
91impl<F> DeploymentRosterSource for F
92where
93    F: Fn() -> Vec<RosterEntry> + Send + Sync,
94{
95    fn roster(&self) -> Arc<Vec<RosterEntry>> {
96        Arc::new(self())
97    }
98}
99
100/// The cached slot: `(refreshed_at, snapshot)`, lock-shared with the
101/// background refresher thread.
102type RosterSlot = parking_lot::Mutex<Option<(std::time::Instant, Arc<Vec<RosterEntry>>)>>;
103
104/// TTL-bounded cached roster projection (the
105/// `resolve_tenant_for_hosted_did` shape): lookups serve a snapshot for at
106/// most [`ROSTER_CACHE_TTL`], then one refresh runs while concurrent lookups
107/// keep serving the current snapshot. After the synchronous first load
108/// (spawn path — data must exist before serving), every refresh runs on a
109/// background thread and publishes atomically: the loader opens RocksDB and
110/// verifies every state, the 100 ms currentness watchdog reaches lookups
111/// from async current-thread runtimes, and public ingestion can amplify the
112/// store — so no runtime thread ever executes the scan. Readers never take
113/// a lock across a scan; the snapshot mutex is touched only long enough to
114/// read or swap an `Arc`.
115///
116/// A refresh that fails loads an EMPTY roster — fail-closed, exactly like an
117/// uncached read failure — and a previously loaded snapshot is never
118/// resurrected after a failure (that would delay revocation beyond the TTL).
119/// Spawn-time checks ride the same cache; at process start it is empty, so
120/// the first lookup is a fresh load.
121pub struct CachedRoster {
122    load: Arc<dyn Fn() -> Vec<RosterEntry> + Send + Sync>,
123    /// Guarded only for the instant of a read or a swap — never across a
124    /// store scan. Shared with the background refresher thread.
125    snapshot: Arc<RosterSlot>,
126    /// Single-flight gate for refreshes. Cleared by [`RefreshGate`]'s Drop,
127    /// which runs even if the refresher panics, so a failed refresh can
128    /// never wedge the cache stale forever.
129    refreshing: Arc<std::sync::atomic::AtomicBool>,
130    ttl: std::time::Duration,
131}
132
133/// Owns the refreshing gate for the duration of one refresh; Drop releases
134/// it on normal exit AND on panic (the background thread's closure drops
135/// its locals when it unwinds).
136struct RefreshGate(Arc<std::sync::atomic::AtomicBool>);
137
138impl Drop for RefreshGate {
139    fn drop(&mut self) {
140        self.0.store(false, std::sync::atomic::Ordering::Release);
141    }
142}
143
144/// How long a roster snapshot is served before the next lookup refreshes it.
145/// Bounds the visibility delay of roster-side changes (churn, reassignment,
146/// squatter arrival); state removal is caught immediately by the uncached
147/// per-DID authority recheck regardless.
148pub const ROSTER_CACHE_TTL: std::time::Duration = std::time::Duration::from_secs(1);
149
150impl CachedRoster {
151    pub fn new(
152        load: Arc<dyn Fn() -> Vec<RosterEntry> + Send + Sync>,
153        ttl: std::time::Duration,
154    ) -> Self {
155        Self {
156            load,
157            snapshot: Arc::new(parking_lot::Mutex::new(None)),
158            refreshing: Arc::new(std::sync::atomic::AtomicBool::new(false)),
159            ttl,
160        }
161    }
162
163    fn fresh(&self) -> Option<Arc<Vec<RosterEntry>>> {
164        let guard = self.snapshot.lock();
165        match guard.as_ref() {
166            Some((refreshed_at, entries)) if refreshed_at.elapsed() <= self.ttl => {
167                Some(Arc::clone(entries))
168            }
169            _ => None,
170        }
171    }
172
173    fn stored(&self) -> Option<Arc<Vec<RosterEntry>>> {
174        self.snapshot.lock().as_ref().map(|(_, e)| Arc::clone(e))
175    }
176
177    fn current(&self) -> Arc<Vec<RosterEntry>> {
178        if let Some(entries) = self.fresh() {
179            return entries;
180        }
181        // Expired (or never loaded). Try to become the refresher without
182        // blocking anyone.
183        if !self
184            .refreshing
185            .swap(true, std::sync::atomic::Ordering::AcqRel)
186        {
187            // Double-check after winning: a refresh may have completed while
188            // we raced to the gate.
189            if let Some(entries) = self.fresh() {
190                self.refreshing
191                    .store(false, std::sync::atomic::Ordering::Release);
192                return entries;
193            }
194            // First-ever load runs synchronously: the spawn-time callers
195            // (self-binding, publisher roster) MUST have data before the
196            // service starts serving, and at this point no sessions exist
197            // that an inline scan could stall.
198            let gate = RefreshGate(Arc::clone(&self.refreshing));
199            if self.stored().is_none() {
200                let entries = Arc::new((self.load)());
201                *self.snapshot.lock() = Some((std::time::Instant::now(), Arc::clone(&entries)));
202                drop(gate);
203                return entries;
204            }
205            // Subsequent refreshes run on a background thread and publish
206            // atomically: the loader opens RocksDB and verifies every state,
207            // and the 100 ms currentness watchdog reaches this from async
208            // current-thread runtimes — an inline scan would stall all
209            // network I/O of the service for the scan duration, once per
210            // TTL, on a store whose size public ingestion can amplify. The
211            // caller keeps the (≤ TTL + one scan) stale snapshot meanwhile.
212            let load = Arc::clone(&self.load);
213            let snapshot = Arc::clone(&self.snapshot);
214            std::thread::spawn(move || {
215                let entries = Arc::new((load)());
216                *snapshot.lock() = Some((std::time::Instant::now(), entries));
217                drop(gate);
218            });
219            // Unreachable in practice (checked stored() above); fail-closed
220            // rather than panic if raced.
221            return self.stored().unwrap_or_else(|| Arc::new(Vec::new()));
222        }
223        // Another refresh is in flight: serve the current snapshot rather
224        // than block on the scan. Only the never-loaded case (synchronous
225        // first load at spawn) waits, bounded.
226        if let Some(entries) = self.stored() {
227            return entries;
228        }
229        let deadline = std::time::Instant::now() + std::time::Duration::from_secs(30);
230        while self.stored().is_none() {
231            if !self.refreshing.load(std::sync::atomic::Ordering::Acquire) {
232                break;
233            }
234            if std::time::Instant::now() > deadline {
235                // Fail-closed, like every other unavailable-roster path.
236                return Arc::new(Vec::new());
237            }
238            std::thread::sleep(std::time::Duration::from_millis(5));
239        }
240        self.stored().unwrap_or_else(|| Arc::new(Vec::new()))
241    }
242}
243
244impl DeploymentRosterSource for CachedRoster {
245    fn roster(&self) -> Arc<Vec<RosterEntry>> {
246        self.current()
247    }
248}
249
250/// The deployment service name an accepted state proves, if any.
251///
252/// Capsule `#<name>` service entries ∩ the registered factory roster
253/// (`hyprstream_service::get_factory`), which must match exactly ONE service,
254/// and the state must carry the deployment invariants (successor epoch > 0
255/// and a bounded expiry — the offline provisioner always stacks a bounded
256/// successor, so genesis-only/unbounded states are not deployment-shaped).
257/// Zero matches — foreign `#id` entries, no service entries — ambiguous
258/// matches, and invariant-violating states all resolve to `None`: deny,
259/// never a best-effort pick.
260pub fn derived_service_name(state: &AcceptedIdentityState, now_unix_ms: i64) -> Option<&str> {
261    // Live expiry is enforced here, not just presence: an expired state is
262    // rejected by the admission exchange, and it must not keep participating
263    // in name derivation or uniqueness counting either — otherwise an
264    // expired foreign claim would make the genuine identity ambiguously
265    // denied forever (review round 2).
266    if state.epoch == 0 || state.expires_at_unix_ms.is_none() || !state.is_live(now_unix_ms) {
267        return None;
268    }
269    let mut matched = None;
270    for id in &state.service_ids {
271        let Some(name) = id.strip_prefix('#') else {
272            continue;
273        };
274        if hyprstream_service::get_factory(name).is_some() {
275            if matched.is_some() {
276                return None;
277            }
278            matched = Some(name);
279        }
280    }
281    matched
282}
283
284/// Resolve `did` to its deployment service name against the live roster,
285/// enforcing unique holdership: the DID's state must derive a name, and no
286/// OTHER state may derive the same name. A collision is warned loudly —
287/// an ingested record squatting a deployment name is an operator-visible
288/// event, not a silent deny.
289pub fn roster_service_name(roster: &[RosterEntry], did: &str, now_unix_ms: i64) -> Option<String> {
290    let name = derived_service_name(&roster.iter().find(|(d, _)| d == did)?.1, now_unix_ms)?;
291    let holders = roster
292        .iter()
293        .filter(|(_, state)| derived_service_name(state, now_unix_ms) == Some(name))
294        .count();
295    if holders != 1 {
296        tracing::warn!(
297            service = name,
298            holders,
299            "accepted-state store holds multiple identities deriving service; denying until unique"
300        );
301        return None;
302    }
303    Some(name.to_owned())
304}
305
306/// DID → derived tenant against the live roster: [`LOCAL_TENANT`] iff the
307/// subject uniquely proves a deployment service identity.
308pub fn roster_tenant(roster: &[RosterEntry], did: &str, now_unix_ms: i64) -> Option<String> {
309    roster_service_name(roster, did, now_unix_ms).map(|_| LOCAL_TENANT.to_owned())
310}
311
312/// The admission tenant resolver: tenant [`LOCAL_TENANT`] iff the verified
313/// subject is a current, uniquely-held deployment service identity.
314///
315/// The peer is only ever the ALREADY-VERIFIED admission subject; the tenant is
316/// resolved server-side from live accepted state, never read from the peer or
317/// its proof. Re-ridden by the live-session recheck, so roster removal,
318/// service-name reassignment, or a new ambiguous squatter closes previously
319/// admitted sessions.
320pub fn derived_tenant_resolver(roster: Arc<dyn DeploymentRosterSource>) -> PeerTenantResolver {
321    Arc::new(move |peer: &PeerIdentity| {
322        peer.subject.as_deref().and_then(|did| {
323            let snapshot = roster.roster();
324            roster_tenant(
325                &snapshot,
326                did,
327                hyprstream_rpc::envelope::current_timestamp(),
328            )
329        })
330    })
331}
332
333/// Spawn-time self-binding (#1652): the live roster must resolve the
334/// process's OWN admission proof to exactly this service. Refusal propagates
335/// and the service never spawns — the fail-closed shape of the former
336/// empty-tenant-map refusal, now "the checkpoint store must bind my DID to my
337/// name". The store is guaranteed present at spawn (the accepted-state source
338/// refuses to open without it), so failure here is a provisioning mismatch,
339/// not a degraded mode to run in.
340pub fn require_service_self_binding(
341    roster: &Arc<dyn DeploymentRosterSource>,
342    service_name: &str,
343    proof: &MoqlAdmissionProof,
344) -> Result<()> {
345    let snapshot = roster.roster();
346    let resolved = roster_service_name(
347        &snapshot,
348        &proof.did,
349        hyprstream_rpc::envelope::current_timestamp(),
350    );
351    anyhow::ensure!(
352        resolved.as_deref() == Some(service_name),
353        "native {} identity {} does not uniquely resolve to accepted deployment service '{}' in the checkpoint store",
354        service_name,
355        proof.did,
356        service_name
357    );
358    Ok(())
359}
360
361/// Client-side membership self-check: the process's own proof must resolve to
362/// SOME uniquely-held deployment service. CLI/embedded processes bootstrap
363/// with one of the deployment identities (whichever the process was
364/// provisioned as), so — like the former `require_event_tenant`, which
365/// accepted any local-bound DID — the check is roster membership, not
366/// equality with a specific service.
367pub fn require_admitted_service_binding(
368    roster: &Arc<dyn DeploymentRosterSource>,
369    proof: &MoqlAdmissionProof,
370) -> Result<()> {
371    let snapshot = roster.roster();
372    anyhow::ensure!(
373        roster_service_name(&snapshot, &proof.did, hyprstream_rpc::envelope::current_timestamp())
374            .is_some(),
375        "Event identity {} is not a uniquely-held accepted deployment service identity in the checkpoint store",
376        proof.did
377    );
378    Ok(())
379}
380
381/// Spawn-time Event publisher roster check (#1652): every name in
382/// `quic.event_publishers` must be carried by EXACTLY ONE live accepted
383/// state — the same unique-holder rule the runtime resolution enforces, so
384/// spawn fails loudly instead of starting with a publisher that every lookup
385/// would deny (a name with no holder is a deployment misconfiguration —
386/// typo, decommissioned service; a name with two holders is an ingested
387/// squatter beside the genuine identity). It does NOT establish provenance:
388/// a sole foreign claimant of an unheld name satisfies it exactly as it
389/// satisfies the derivation, which is the maintainer provenance-binding
390/// decision recorded in #1652 (see the module doc residual); closing that
391/// here is not possible from store data alone.
392pub fn require_publisher_roster(
393    roster: &Arc<dyn DeploymentRosterSource>,
394    publishers: &std::collections::BTreeSet<String>,
395) -> Result<()> {
396    let live = roster.roster();
397    let now = hyprstream_rpc::envelope::current_timestamp();
398    for name in publishers {
399        let holders = live
400            .iter()
401            .filter(|(_, state)| derived_service_name(state, now) == Some(name.as_str()))
402            .count();
403        anyhow::ensure!(
404            holders == 1,
405            "quic.event_publishers lists '{}' but {} live accepted states derive that service (runtime resolution requires exactly one); refusing to spawn",
406            name,
407            holders
408        );
409    }
410    Ok(())
411}
412
413#[cfg(test)]
414mod tests {
415    #![allow(clippy::expect_used, clippy::unwrap_used)]
416    use super::*;
417    use hyprstream_rpc::transport::moql_admission::AcceptedSubjectKey;
418    use std::collections::BTreeMap;
419
420    /// The factory inventory (`hyprstream_service::get_factory`) is populated
421    /// by `#[service_factory]` submissions in the APP crate, which this crate
422    /// cannot link (the app depends on us). This test binary therefore
423    /// registers its own admission-only roster: factory functions that are
424    /// never called — `derived_service_name` consults only the names. The
425    /// submissions are visible to every test in this binary, including the
426    /// `moql_checkpoint_admission_tests` fixtures, whose capsule service
427    /// entries name "streams" and "reader".
428    fn admission_only_test_factory(
429        _ctx: &hyprstream_service::ServiceContext,
430    ) -> anyhow::Result<Box<dyn hyprstream_service::Spawnable>> {
431        anyhow::bail!("admission-only test factory is never spawned")
432    }
433
434    inventory::submit! {
435        hyprstream_service::ServiceFactory::new("event", admission_only_test_factory)
436    }
437    inventory::submit! {
438        hyprstream_service::ServiceFactory::new("streams", admission_only_test_factory)
439    }
440    inventory::submit! {
441        hyprstream_service::ServiceFactory::new("model", admission_only_test_factory)
442    }
443    inventory::submit! {
444        hyprstream_service::ServiceFactory::new("policy", admission_only_test_factory)
445    }
446    inventory::submit! {
447        hyprstream_service::ServiceFactory::new("reader", admission_only_test_factory)
448    }
449
450    fn state(service_ids: &[&str]) -> AcceptedIdentityState {
451        AcceptedIdentityState {
452            epoch: 1,
453            head_digest: [1; 64],
454            subject_keys: vec![AcceptedSubjectKey {
455                ed25519: [2; 32],
456                ml_dsa_65: vec![3; 1952],
457            }],
458            service_ids: service_ids.iter().map(|id| (*id).to_owned()).collect(),
459            expires_at_unix_ms: Some(i64::MAX),
460        }
461    }
462
463    fn peer(did: &str) -> PeerIdentity {
464        PeerIdentity::authenticated(did.to_owned())
465    }
466
467    #[test]
468    fn factory_service_entries_resolve_and_foreign_entries_do_not() {
469        let now = hyprstream_rpc::envelope::current_timestamp();
470        // A deployment-shaped capsule: exactly one factory service entry,
471        // successor epoch, bounded unexpired expiry.
472        assert_eq!(
473            derived_service_name(&state(&["#event"]), now),
474            Some("event")
475        );
476        assert_eq!(
477            derived_service_name(&state(&["#streams"]), now),
478            Some("streams")
479        );
480        // Foreign public-RPC records carry arbitrary #id entries.
481        assert_eq!(derived_service_name(&state(&["#ns"]), now), None);
482        // Unknown factory names never match, with or without foreign siblings.
483        assert_eq!(derived_service_name(&state(&["#bogus"]), now), None);
484        assert_eq!(derived_service_name(&state(&["#ns", "#bogus"]), now), None);
485        // No service entries at all.
486        assert_eq!(derived_service_name(&state(&[]), now), None);
487        // Ambiguous: a capsule matching two factory services is not
488        // deployment-shaped; deny rather than pick.
489        assert_eq!(
490            derived_service_name(&state(&["#event", "#streams"]), now),
491            None
492        );
493        // Non-# entries are skipped, not matched.
494        assert_eq!(derived_service_name(&state(&["event"]), now), None);
495        // Deployment invariants: genesis (epoch 0) and unbounded states never
496        // derive — foreign genesis-only ingests are not deployment-shaped.
497        let mut genesis = state(&["#event"]);
498        genesis.epoch = 0;
499        assert_eq!(derived_service_name(&genesis, now), None);
500        let mut unbounded = state(&["#event"]);
501        unbounded.expires_at_unix_ms = None;
502        assert_eq!(derived_service_name(&unbounded, now), None);
503        // EXPIRED states never derive and never count toward uniqueness
504        // (review round 2): an expired foreign claim must not keep the
505        // genuine identity ambiguously denied.
506        let mut expired = state(&["#event"]);
507        expired.expires_at_unix_ms = Some(now - 1);
508        assert_eq!(derived_service_name(&expired, now), None);
509    }
510
511    #[test]
512    fn tenant_resolver_derives_local_only_from_live_state() {
513        let did = "did:at9p:abc";
514        let foreign = "did:at9p:foreign";
515        let states: Arc<parking_lot::RwLock<BTreeMap<String, AcceptedIdentityState>>> =
516            Arc::new(parking_lot::RwLock::new(BTreeMap::from([
517                (did.to_owned(), state(&["#event"])),
518                (foreign.to_owned(), state(&["#ns"])),
519            ])));
520        let roster: Arc<dyn DeploymentRosterSource> = {
521            let states = Arc::clone(&states);
522            Arc::new(move || states.read().clone().into_iter().collect::<Vec<_>>())
523        };
524        let resolver = derived_tenant_resolver(Arc::clone(&roster));
525        assert_eq!(resolver(&peer(did)).as_deref(), Some(LOCAL_TENANT));
526        // Foreign record present in the store but not a factory service: deny.
527        assert_eq!(resolver(&peer(foreign)), None);
528        // Unknown DID: deny.
529        assert_eq!(resolver(&peer("did:at9p:unknown")), None);
530
531        // THE CHURN CASE (#1652): the same service name is re-bound to a NEW
532        // DID in the live store (as at an R1 store re-init). Admission for the
533        // new DID works in the same process, with no restart and no config —
534        // impossible under the former DID-keyed config map.
535        let successor = "did:at9p:successor";
536        states.write().remove(did);
537        states
538            .write()
539            .insert(successor.to_owned(), state(&["#event"]));
540        assert_eq!(
541            resolver(&peer(did)),
542            None,
543            "superseded DID must stop resolving"
544        );
545        assert_eq!(
546            resolver(&peer(successor)).as_deref(),
547            Some(LOCAL_TENANT),
548            "successor DID must resolve without restart or config change"
549        );
550    }
551
552    #[test]
553    fn name_squatting_is_denied_not_escalated() {
554        let now = hyprstream_rpc::envelope::current_timestamp();
555        // A foreign record claiming a factory service name BESIDE the genuine
556        // identity: the name becomes ambiguous and BOTH derive nothing —
557        // fail-closed. This is the property a name-only filter lacks.
558        let genuine = "did:at9p:genuine-model";
559        let squatter = "did:at9p:foreign-model";
560        let roster: Vec<RosterEntry> = vec![
561            (genuine.to_owned(), state(&["#model"])),
562            (squatter.to_owned(), state(&["#model"])),
563        ];
564        assert_eq!(roster_tenant(&roster, genuine, now), None);
565        assert_eq!(roster_tenant(&roster, squatter, now), None);
566        // With the squatter gone the genuine identity derives again.
567        let clean: Vec<RosterEntry> = vec![(genuine.to_owned(), state(&["#model"]))];
568        assert_eq!(
569            roster_tenant(&clean, genuine, now).as_deref(),
570            Some(LOCAL_TENANT)
571        );
572        // An EXPIRED squatter never counts: the genuine identity keeps
573        // deriving while an expired foreign claim sits in the store.
574        let mut expired_squat = state(&["#model"]);
575        expired_squat.expires_at_unix_ms = Some(now - 1);
576        let with_expired: Vec<RosterEntry> = vec![
577            (genuine.to_owned(), state(&["#model"])),
578            (squatter.to_owned(), expired_squat),
579        ];
580        assert_eq!(
581            roster_tenant(&with_expired, genuine, now).as_deref(),
582            Some(LOCAL_TENANT),
583            "expired foreign claim must not make the genuine identity ambiguous"
584        );
585    }
586
587    #[test]
588    fn store_read_failure_denies_like_unknown_identity() {
589        // The production roster maps source errors to an empty roster; a
590        // resolver over such a source inherits deny-all, exactly like
591        // UnknownIdentity.
592        let roster: Arc<dyn DeploymentRosterSource> = Arc::new(Vec::new);
593        let resolver = derived_tenant_resolver(roster);
594        assert_eq!(resolver(&peer("did:at9p:anything")), None);
595    }
596
597    #[test]
598    fn self_binding_requires_the_services_own_name() {
599        let did = "did:at9p:streams";
600        let roster: Arc<dyn DeploymentRosterSource> =
601            Arc::new(move || vec![(did.to_owned(), state(&["#streams"]))]);
602        let proof = MoqlAdmissionProof {
603            did: did.to_owned(),
604            ed25519: ed25519_dalek::SigningKey::from_bytes(&[4; 32]),
605            ml_dsa_65: hyprstream_rpc::crypto::pq::ml_dsa_sk_from_seed(&[5; 32]),
606            expected_server: hyprstream_rpc::stream_info::MoqlServerIdentity {
607                did: did.to_owned(),
608                epoch: 1,
609                head_digest: vec![1; 64],
610                expires_at_unix_ms: i64::MAX,
611                ed25519: [2; 32],
612                ml_dsa65: vec![3; 1952],
613            },
614        };
615        require_service_self_binding(&roster, "streams", &proof)
616            .expect("own service name must bind");
617        let mismatch = require_service_self_binding(&roster, "event", &proof)
618            .expect_err("a different service name must refuse");
619        assert!(
620            mismatch.to_string().contains("does not uniquely resolve"),
621            "unexpected error: {mismatch}"
622        );
623        // Client-side membership form accepts the deployment identity itself.
624        require_admitted_service_binding(&roster, &proof)
625            .expect("membership self-check must accept a deployment service DID");
626    }
627
628    #[test]
629    fn cached_roster_serves_within_ttl_and_refreshes_after() {
630        let states: Arc<parking_lot::RwLock<Vec<RosterEntry>>> = Arc::new(
631            parking_lot::RwLock::new(vec![("did:at9p:event".to_owned(), state(&["#event"]))]),
632        );
633        let load = {
634            let states = Arc::clone(&states);
635            Arc::new(move || states.read().clone())
636                as Arc<dyn Fn() -> Vec<RosterEntry> + Send + Sync>
637        };
638        let now = hyprstream_rpc::envelope::current_timestamp();
639        // Zero TTL: the first lookup loads synchronously; later refreshes
640        // run in the background and publish — churn becomes visible after at
641        // most one async load (bounded poll here; the loader is instant).
642        let hot = CachedRoster::new(Arc::clone(&load), std::time::Duration::ZERO);
643        assert_eq!(
644            roster_tenant(&hot.roster(), "did:at9p:event", now).as_deref(),
645            Some(LOCAL_TENANT)
646        );
647        states.write().clear();
648        let deadline = std::time::Instant::now() + std::time::Duration::from_secs(5);
649        while !hot.roster().is_empty() {
650            assert!(
651                std::time::Instant::now() < deadline,
652                "zero-TTL cache never observed the wipe"
653            );
654            std::thread::sleep(std::time::Duration::from_millis(5));
655        }
656        // Long TTL: the snapshot is served as-is until it expires — the
657        // polling recheck path trades bounded staleness for not re-verifying
658        // the whole store at 10 Hz per session.
659        states
660            .write()
661            .push(("did:at9p:event".to_owned(), state(&["#event"])));
662        let cold = CachedRoster::new(Arc::clone(&load), std::time::Duration::from_secs(3600));
663        assert_eq!(cold.roster().len(), 1);
664        states.write().clear();
665        assert_eq!(
666            cold.roster().len(),
667            1,
668            "long-TTL cache must keep serving the snapshot"
669        );
670    }
671
672    /// A refresh in progress must not block concurrent lookups, and after
673    /// the first load the scan never runs inline on the caller: the caller
674    /// keeps serving the current snapshot while a background thread loads
675    /// and publishes atomically (review rounds 4 and 5).
676    #[test]
677    fn cached_roster_readers_never_block_and_refreshes_run_in_background() {
678        let calls = Arc::new(std::sync::atomic::AtomicUsize::new(0));
679        let load = {
680            let calls = Arc::clone(&calls);
681            Arc::new(move || {
682                std::thread::sleep(std::time::Duration::from_millis(300));
683                let n = calls.fetch_add(1, std::sync::atomic::Ordering::SeqCst) + 1;
684                vec![("did:at9p:event".to_owned(), state(&["#event"])); n]
685            }) as Arc<dyn Fn() -> Vec<RosterEntry> + Send + Sync>
686        };
687        let roster = Arc::new(CachedRoster::new(load, std::time::Duration::from_millis(1)));
688        // First load is synchronous (spawn-path semantics): one call, len 1.
689        assert_eq!(roster.current().len(), 1);
690        assert_eq!(calls.load(std::sync::atomic::Ordering::SeqCst), 1);
691        std::thread::sleep(std::time::Duration::from_millis(10)); // TTL lapses
692                                                                  // Expired lookup schedules the refresh on a background thread and
693                                                                  // returns the stale snapshot IMMEDIATELY — the 300 ms scan must not
694                                                                  // run on this (simulated runtime) thread.
695        let started = std::time::Instant::now();
696        let served = roster.current();
697        assert_eq!(served.len(), 1, "caller must serve the stale snapshot");
698        assert!(
699            started.elapsed() < std::time::Duration::from_millis(200),
700            "expired lookup ran the slow scan inline"
701        );
702        // A concurrent lookup during the in-flight refresh also serves the
703        // stale snapshot without blocking.
704        std::thread::sleep(std::time::Duration::from_millis(50));
705        let started = std::time::Instant::now();
706        assert_eq!(roster.current().len(), 1);
707        assert!(
708            started.elapsed() < std::time::Duration::from_millis(100),
709            "concurrent lookup blocked on the in-flight refresh scan"
710        );
711        // The background refresh publishes; the next lookup observes it.
712        let deadline = std::time::Instant::now() + std::time::Duration::from_millis(1000);
713        while roster.current().len() != 2 {
714            assert!(
715                std::time::Instant::now() < deadline,
716                "background refresh never published"
717            );
718            std::thread::sleep(std::time::Duration::from_millis(20));
719        }
720        assert_eq!(calls.load(std::sync::atomic::Ordering::SeqCst), 2);
721    }
722
723    #[test]
724    fn publisher_roster_refuses_names_without_a_live_identity() {
725        let did = "did:at9p:event";
726        let roster: Arc<dyn DeploymentRosterSource> =
727            Arc::new(move || vec![(did.to_owned(), state(&["#event"]))]);
728        require_publisher_roster(
729            &roster,
730            &["event".to_owned(), "model".to_owned()]
731                .into_iter()
732                .collect(),
733        )
734        .expect_err("a configured publisher with no live state must refuse spawn");
735        require_publisher_roster(&roster, &["event".to_owned()].into_iter().collect())
736            .expect("a uniquely carried publisher name must pass");
737        // Ambiguous holders (genuine identity + ingested squatter both
738        // deriving the configured name) must refuse spawn: runtime lookups
739        // would deny both, so a passing check would advertise a publisher
740        // that cannot work (review round 7).
741        let did = "did:at9p:event";
742        let ambiguous: Arc<dyn DeploymentRosterSource> = Arc::new(move || {
743            vec![
744                (did.to_owned(), state(&["#event"])),
745                ("did:at9p:squatter".to_owned(), state(&["#event"])),
746            ]
747        });
748        require_publisher_roster(&ambiguous, &["event".to_owned()].into_iter().collect())
749            .expect_err("an ambiguously held publisher name must refuse spawn");
750    }
751}